Add tests for Angular and Node.js webapp in Azure Pipelines (part 3)

Joost Luijben

In the previous article I showed how to build a webapp in the Azure pipeline. In this article I will explain to you how you run tests in your pipeline. I will show you how to do this for both Angular and Node.js.

Adding tests frontend – Angular

Adding tests to the frontend is a bit harder than for the backend, so will we start out with this.
This article helped me out a lot by adding tests to Angular. I will cover what Oliver said in that article and add some more about how to improve performance.

Adding code coverage

First we will want to add the Cobertura reporter to karma.conf.js by doing the following

module.exports = function (config) {
...
    coverageIstanbulReporter: {
      dir: require('path').join(__dirname, '../coverage'),
      reports: ['html', 'lcovonly', 'text-summary', 'cobertura'],
      fixWebpackSourcePaths: true
    },
...
};

Next you will want to install the following three packages as dev dependencies, so the tests can run headless.

  • Puppeteer
  • Karma-jasmine-html-reporter
  • Karma-spec-reporter

Now you will want to edit the karma.conf.js again so it uses puppeteer. Make it so it looks something like this. The highlighted line are the lines that have been changed.

module.exports = function (config) {
  const puppeteer = require('puppeteer');
  process.env.CHROME_BIN = puppeteer.executablePath();

  config.set({
    basePath: '',
    frameworks: ['jasmine', '@angular-devkit/build-angular'],
    plugins: [
      require('karma-jasmine'),
      require('karma-chrome-launcher'),
      require('karma-jasmine-html-reporter'),
      require('karma-coverage-istanbul-reporter'),
      require('@angular-devkit/build-angular/plugins/karma')
    ],
    client: {
      clearContext: false // leave Jasmine Spec Runner output visible in browser
    },
    coverageIstanbulReporter: {
      dir: require('path').join(__dirname, '../coverage'),
      reports: ['html', 'lcovonly', 'text-summary', 'cobertura'],
      fixWebpackSourcePaths: true
    },
    reporters: ['progress', 'kjhtml'],
    port: 9876,
    colors: true,
    logLevel: config.LOG_INFO,
    autoWatch: true,
    browsers: ['ChromeHeadless'],
    singleRun: false
  });
};

The code coverage problem

For the code coverage, we want the backend as well as the frontend reported. The problem is, that azure overrides the code coverage files when a new coverage file is uploaded. See here. Now there’s a solution for that (until Microsoft fixes this issue). We can use an custom reporter made by a guy named Daniel palm. This reporter can merge files, in contrary to Azure’s reporter.
You can install his report generator from the marketplace from here. Now you can use it in your pipeline.
Run the tests and create the code coverage file like so:

- task: DeleteFiles@1
  displayName: 'Delete JUnit files'
  inputs:
    SourceFolder: '$(System.DefaultWorkingDirectory)/frontend/junit'
    Contents: 'TESTS*.xml'
- task: Npm@1
  displayName: 'Test Angular'
  inputs:
    command: custom
    customCommand: run test -- --watch=false --code-coverage
    workingDir: '$(System.DefaultWorkingDirectory)/frontend'
- task: Palmmedia.reportgenerator.reportgenerator-build-release-task.reportgenerator@4
  displayName: ReportGenerator
  inputs:
    reports: '$(Build.SourcesDirectory)/frontend/coverage/cobertura-coverage.xml'
    targetdir: '$(Build.SourcesDirectory)/frontend/coverage'
    reporttypes: 'Cobertura'
    assemblyfilters: '-xunit*'
- task: PublishPipelineArtifact@1
  inputs:
    targetPath: '$(Build.SourcesDirectory)/frontend/coverage'
    artifact: 'frontendCoverage'

We’re first running a deleteFiles task, to make sure the leftover files from the previous build don’t mess up the new results.
Next we’re publishing the coverage as an artifact (yes this is just one file), so we can use it later on when we’re publishing the codeCoverage.

Adding the test summary

For the tests we will just use the ng test command. We’re going to use a junit reporter. Install it by running

npm install karma-junit-reporter --save-dev

Next, add the junit reporter to the karma config by adding the following to the karma.conf.js:

require('karma-jasmine'),
require('karma-chrome-launcher'),
require('karma-jasmine-html-reporter'),
require('karma-coverage-istanbul-reporter'),
require('@angular-devkit/build-angular/plugins/karma'),
require('karma-junit-reporter')
reporters: ['progress', 'kjhtml', 'junit'],
junitReporter: {
    outputDir: './junit'
},

Last, add the tests results to the pipeline by adding this to the azure-pipelines.yml:

- task: PublishTestResults@2
  displayName: 'Publish Angular test results'
  condition: succeededOrFailed()
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: '$(System.DefaultWorkingDirectory)/frontend/junit/TESTS-*.xml'
    searchFolder: '$(System.DefaultWorkingDirectory)/frontend/junit'
    testRunTitle: 'Frontend mocha tests'

Improving performance

In my experience, the performance of the tests is very, very slow. For me it sometimes takes up to two whole minutes to complete the tests. Sadly, this is mainly caused by the angular compiler itself. There are some ways to reduce the build time a bit, but for me it didn’t help much. The test time was still reduced by 20 seconds though. So you might consider it.
This article explains very well how why the tests are so slow in detail. It also explains how to solve this. This article also shows some ways to speed the process up.
Basically the main thing is that for every test, Angular creates an TestBed. You can also instantiate this TestBed just one time and run it for every test. In the provided article zshuffield used Ng-Bullet. You can use Ng-bullet by installing it and changing the test code to look like the following:

ng-bullet component

The disadvantage of creating a TestBed just one time, is that code can mess up at runtime. I would only recommend trying to increase the test time if you have a lot of tests. Think of thousands of tests.

Adding tests backend – Node.js

Adding tests to the backend is considerably easier than for the frontend. To accomplish this I used the answer from crimbo from this stackoverflow post.
If you’ve not installed jest yet, do so now and add some tests.
Then install the jest-junit package in the backend folder

npm add --save-dev jest-junit

Adding code coverage

We will make a script in package.json that runs the test coverage. Add the following under scripts:

"test-azure-coverage": "jest --coverage --coverageReporters=cobertura --forceExit"

Now add the reporters to the jest config. Check the documentation here.

  "jest": {
    "reporters": [
      "default",
      "jest-junit"
    ],

Next you add the script, run the reporter (the palm reporter), add it to the pipeline and publish it. You can do this by adding the following to the azure-pipelines.yml:

- task: Npm@1
  displayName: run test backend coverage
  inputs:
    command: 'custom'
    workingDir: 'backend'
    customCommand: 'run test-azure-coverage'
- task: Palmmedia.reportgenerator.reportgenerator-build-release-task.reportgenerator@4
  displayName: ReportGenerator
  inputs:
    reports: '$(Build.SourcesDirectory)/backend/coverage/cobertura-coverage.xml'
    targetdir: '$(Build.SourcesDirectory)/backend/coverage'
    reporttypes: 'Cobertura'
    assemblyfilters: '-xunit*'
- task: PublishPipelineArtifact@1
  inputs:
    targetPath: '$(Build.SourcesDirectory)/backend/coverage'
    artifact: 'backendCoverage'

Adding the test summary

To add the test summary we again add a script to the package.json

"test-azure-report": "jest --ci --reporters=default --reporters=jest-junit --forceExit",

This should run the summary. The ci flag is for the continuous integration. I added the forceExit, so it exits when it’s done. See here for the documentation

Next add the test command to the pipeline and publish the result by adding the following to the azure-pipelines.yml:

- task: Npm@1
  displayName: run test backend report
  inputs:
    command: 'custom'
    workingDir: 'backend'
    customCommand: 'run test-azure-report'
- task: PublishTestResults@2
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: 'junit.xml'
    searchFolder: '$(Build.SourcesDirectory)/backend'
    testRunTitle: 'Run backend jest tests'

Publishing the code coverage

We’ve published both the frontend and the backend as an artifact. Now we’re going to use these artifacts to publish the coverage altogether.

We do so by adding the following steps to a new job in the azure-pipelines.yml:

- job: codeCoverage
  displayName: publish code coverage
  dependsOn:
   - Frontend
   - Backend
  steps:
    - checkout: none
    - task: DownloadPipelineArtifact@2
      inputs:
        artifact: backend        
        path: $(Pipeline.Workspace)/backend/backend
        itemPattern: '!node_modules/**'
    - task: DownloadPipelineArtifact@2
      inputs:
        artifact: frontend
        path: $(Pipeline.Workspace)/frontend/frontend
        itemPattern: '!node_modules/**'
    - download: current
      artifact: backendCoverage
    - download: current
      artifact: frontendCoverage
    - task: Palmmedia.reportgenerator.reportgenerator-build-release-task.reportgenerator@4
      displayName: ReportGenerator
      inputs:
        reports: '$(Pipeline.Workspace)/backendCoverage/Cobertura.xml;$(Pipeline.Workspace)/frontendCoverage/Cobertura.xml'
        targetdir: '$(Pipeline.Workspace)/coverage'
        reporttypes: 'HtmlInline_AzurePipelines;Cobertura'
    - task: PublishCodeCoverageResults@1
      inputs:
        codeCoverageTool: 'Cobertura'
        summaryFileLocation: '$(Pipeline.Workspace)/coverage/*.xml'
        reportDirectory: '$(Pipeline.Workspace)/coverage'

This job downloads the two source code artifacts (this is needed for reference in the code coverage report later on) and downloads the two generated report files.
Notice how I don’t download the node_modules folder, because this is not necessary and is a very large folder.
We use the report generator job from Palm again to generate the final report in html. We publish this report with the PublishCodeCoverageResults task. Your result after running the pipeline should look something like this:

Notice the 2x CoberturaParser by the code coverage report in the azure pipeline

Notice the (2x) at the parser, this indicates that it has combined two reports into 1 report.

And we’re done! This should add backend and frontend tests and code coverage to your Azure pipeline. In the next article I will explain how you can combine on-premise VM’s with the multiple stage environment, so you can have your environments set up so deployments can only be done by manual approval.

Your code in the end should now look like this. I highlighted the lines that have been added compared to the previous file. Don’t forgot to make the changes to the other files as well!

trigger:
  - master
  
pool:
  vmImage: 'ubuntu-latest'
  
stages:
  - stage: Build
    jobs:
      - job: Frontend
        steps:
          - task: Npm@1
            inputs:
              command: 'ci'
              workingDir: '$(System.DefaultWorkingDirectory)/frontend'
          - task: Bash@3
            inputs:
              targetType: 'inline'
              script: 'npm run ng build'
              workingDirectory: '$(System.DefaultWorkingDirectory)/frontend'
          - task: DeleteFiles@1
            displayName: 'Delete JUnit files'
            inputs:
              SourceFolder: '$(System.DefaultWorkingDirectory)/frontend/junit'
              Contents: 'TESTS*.xml'
          - task: Npm@1
            displayName: 'Test Angular'
            inputs:
              command: custom
              customCommand: run test -- --watch=false --code-coverage
              workingDir: '$(System.DefaultWorkingDirectory)/frontend'
          - task: Palmmedia.reportgenerator.reportgenerator-build-release-task.reportgenerator@4
            displayName: ReportGenerator
            inputs:
              reports: '$(Build.SourcesDirectory)/frontend/coverage/cobertura-coverage.xml'
              targetdir: '$(Build.SourcesDirectory)/frontend/coverage'
              reporttypes: 'Cobertura'
              assemblyfilters: '-xunit*'
          - task: PublishTestResults@2
            displayName: 'Publish Angular test results'
            condition: succeededOrFailed()
            inputs:
              testResultsFormat: 'JUnit'
              testResultsFiles: '$(System.DefaultWorkingDirectory)/frontend/junit/TESTS-*.xml'
              searchFolder: '$(System.DefaultWorkingDirectory)/frontend/junit'
              testRunTitle: 'Frontend mocha tests'
          - task: PublishPipelineArtifact@1
            inputs:
              targetPath: '$(Build.SourcesDirectory)/frontend/coverage'
              artifact: 'frontendCoverage'
          - task: PublishPipelineArtifact@1
            inputs:
              targetPath: '$(Pipeline.Workspace)/frontend'
              artifact: 'frontend'
              publishLocation: 'pipeline'


      - job: Backend
        steps:
          - task: Npm@1
            inputs:
              command: 'ci'
              workingDir: '$(System.DefaultWorkingDirectory)/backend'
          - task: Bash@3
            inputs:
              targetType: 'inline'
              script: 'npm run build'
              workingDirectory: '$(System.DefaultWorkingDirectory)/backend'
          - task: Npm@1
            displayName: run test backend report
            inputs:
              command: 'custom'
              workingDir: 'backend'
              customCommand: 'run test-azure-report'
          - task: Npm@1
            displayName: run test backend coverage
            inputs:
              command: 'custom'
              workingDir: 'backend'
              customCommand: 'run test-azure-coverage'
          - task: Palmmedia.reportgenerator.reportgenerator-build-release-task.reportgenerator@4
            displayName: ReportGenerator
            inputs:
              reports: '$(Build.SourcesDirectory)/backend/coverage/cobertura-coverage.xml'
              targetdir: '$(Build.SourcesDirectory)/backend/coverage'
              reporttypes: 'Cobertura'
              assemblyfilters: '-xunit*'
          - task: PublishTestResults@2
            inputs:
              testResultsFormat: 'JUnit'
              testResultsFiles: 'junit.xml'
              searchFolder: '$(Build.SourcesDirectory)/backend'
              testRunTitle: 'Run backend jest tests'
          - task: PublishPipelineArtifact@1
            inputs:
              targetPath: '$(Build.SourcesDirectory)/backend/coverage'
              artifact: 'backendCoverage'
          - task: PublishPipelineArtifact@1
            inputs:
              targetPath: '$(Pipeline.Workspace)/backend'
              artifact: 'backend'
              publishLocation: 'pipeline'


      - job: codeCoverage
        displayName: publish code coverage
        dependsOn:
          - Frontend
          - Backend
        steps:
          - checkout: none
          - task: DownloadPipelineArtifact@2
            inputs:
              artifact: backend        
              path: $(Pipeline.Workspace)/backend/backend
              itemPattern: '!node_modules/**'
          - task: DownloadPipelineArtifact@2
            inputs:
              artifact: frontend
              path: $(Pipeline.Workspace)/frontend/frontend
              itemPattern: '!node_modules/**'
          - download: current
            artifact: backendCoverage
          - download: current
            artifact: frontendCoverage
          - task: Palmmedia.reportgenerator.reportgenerator-build-release-task.reportgenerator@4
            displayName: ReportGenerator
            inputs:
              reports: '$(Pipeline.Workspace)/backendCoverage/Cobertura.xml;$(Pipeline.Workspace)/frontendCoverage/Cobertura.xml'
              targetdir: '$(Pipeline.Workspace)/coverage'
              reporttypes: 'HtmlInline_AzurePipelines;Cobertura'
          - task: PublishCodeCoverageResults@1
            inputs:
              codeCoverageTool: 'Cobertura'
              summaryFileLocation: '$(Pipeline.Workspace)/coverage/*.xml'
              reportDirectory: '$(Pipeline.Workspace)/coverage'

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Next Post

AWS Shop: about the AWS Simple Notification Service (SNS)

Facebook 0 Twitter Linkedin Introduction Today [1] we’ll look at the AWS Simple Notification Service. We have two of them in our shop: one to get messages from the accept-lambda function and send them to the decrypt-Lambda function, and the decrypt-Lambda function will send the decrypted sales information via SNS […]