Code Coverage with AltCover: How to test .Net Code in Azure DevOps YAML pipelines (2024)

How to run code coverage for .Net code with Ubuntu (Linux) agents? Coverlet works with Windows agents but not with Linux agents.

Solution: work with AltCover

Code coverage is an important step in software development cycle to ensure code quality.

Using modern CI/CD tools, this process is now automated and part of YAML pipelines that run in tools such as Azure DevOps.

Typically, windows agents run the pipeline and the code coverage is run by Coverlet (a 3rd party tool) and the result formatted and posted in SonarQube.

Nevertheless, when moving to Linux agents — for better performance — it leads to the report failing to generate.

Fortunately, switching to AltCover instead of Coverlet solves the issue.

Since there is little documentation on this topic, we would like to share with the wider community how we managed to solve this transition.

As the name suggests, it’s an alternative coverage approach. Rather than working by hooking the .net profiling API at run-time, it works by weaving the same sort of extra IL into the assemblies of interest ahead of execution. This means that it should work pretty much everywhere, whatever your platform, so long as the executing process has write access to the results file. You can even mix-and-match between platforms used to instrument and those under test.

Today, I’m exploring AltCover by Steve Gilham. There are coverage tools that use the .NET Profiling API at run-time, instead, AltCover weaves IL for its coverage.

  • Get a detailed Code Coverage report in Azure DevOps
  • Get a full Code Coverage report in SonarQube
Code Coverage with AltCover: How to test.Net Code in Azure DevOps YAML pipelines (2)
Code Coverage with AltCover: How to test.Net Code in Azure DevOps YAML pipelines (3)

Coverlet Process

1. Run .Net Tests and use DataCollectors to output .Net Tests in openCover files

2. Using reportgenerator, take OpenCover and Outputs them into Corbetura files and Html files

3. Corbertura files provide a detailed .Net Coverage report in Azure DevOps

4. Publish OpenCover Files in SonarQube

Code Coverage with AltCover: How to test.Net Code in Azure DevOps YAML pipelines (4)

What are DataCollectors?

Data collectors perform monitoring operations such as collecting code coverage metrics during test execution.

  • Purpose: One of the primary functions of data collectors like Coverlet is to measure code coverage. Code coverage is a metric used to determine which portions of the codebase are exercised by a set of test cases.
  • Coverage Types: Data collectors can measure different types of code coverage, such as statement coverage, branch coverage, and path coverage. These metrics help identify untested or poorly tested code paths.
  • Usage: Developers and testers use code coverage data to identify areas of the codebase that need more testing, ensuring comprehensive test coverage and helping to uncover potential defects.
  • More than 30% faster than windows
  • Different tools available

Overall, Linux agents offer flexibility, cost savings, and efficiency for organizations focused on Linux-based development in Azure DevOps pipelines.

- AltCover runs during compile time

  • With Coverlet, it depends on the Runtime, so if there is a new runtime (e.g. .Net7) then support for Coverlet needs to be updated
Code Coverage with AltCover: How to test.Net Code in Azure DevOps YAML pipelines (5)

- Work of a single individual: Steve Gilham

- Limited Documentation & Code examples

See AltCover GitHub repo: https://github.com/SteveGilham/altcover

- Run .Net Tests then Ouput them in openCover Files

- Use reportgenerator to transform the openCover files in Corbertura

- Cobertura then publishes the Corbertura file in Azure DevOps

  • SonarQube publishes the OpenCover file in SonarQube
Code Coverage with AltCover: How to test.Net Code in Azure DevOps YAML pipelines (6)
- task: SonarQubePrepare@5
displayName: 'SonarQube prepare'
inputs:
SonarQube: 'TS SonarQube EE P'
scannerMode: CLI
cliSources: $(Build.SourcesDirectory)
cliProjectName: nameofcliproject.${{ variables.Name }}'
extraProperties: |
sonar.projectKey=nameofkey
sonar.cs.opencover.reportsPaths=$(Build.SourcesDirectory)/coverage.*.opencover.xml

- task: DotNetCoreCLI@2
displayName: 'Run tests'
inputs:
command: test
projects: |
**/ nameofproject2.Tests.csproj
**/nameofproject1.Tests.csproj
arguments: '/p:AltCover=true /p:AltCoverForce=true /p:AltCoverReport=$(Build.SourcesDirectory)/coverage.$(ProjectName).opencover.xml /p:AltCoverAssemblyExcludeFilter=Tests /p:AltCoverAssemblyFilter=Tests|?(project1|| project2) /p:AltCoverAttributeFilter=ExcludeFromCodeCoverage /p:AltCoverVerbosity=Warning'

- task: DotNetCoreCLI@2
displayName: Restore dotnet tools
inputs:
command: custom
custom: tool
arguments: restore

- script: |
dotnet reportgenerator -reports:$(Build.SourcesDirectory)/coverage.*.opencover.xml '-targetdir:$(Build.SourcesDirectory)/' '-reporttypes:TextSummary;Cobertura'
displayName: Generate report

- task: PublishCodeCoverageResults@1
displayName: 'Publish code coverage report'
inputs:
codeCoverageTool: 'cobertura'
summaryFileLocation: '$(Build.SourcesDirectory)/Cobertura.xml'

- task: SonarQubeAnalyze@5
displayName: 'SonarQube analyze'

- task: SonarQubePublish@5
displayName: 'SonarQube publish results'
inputs:
pollingTimeoutSec: '300'

Code Coverage with AltCover: How to test.Net Code in Azure DevOps YAML pipelines (7)

The * ensures that all files in the directory are included in the code coverage

Result = SonarQube will create reports based on all opencover.xml files found in the mentioned directory

Code Coverage with AltCover: How to test.Net Code in Azure DevOps YAML pipelines (8)

Result = Tests are run and converted by AltCover into opencover.xml files

Full Arguments string:

arguments: ‘/p:AltCover=true /p:AltCoverForce=true /p:AltCoverReport=$(Build.SourcesDirectory)/coverage.$(ProjectName).opencover.xml /p:AltCoverAssemblyExcludeFilter=Tests /p:AltCoverAssemblyFilter=Tests|?(nameofproject1|| nameofproject2||) /p:AltCoverAttributeFilter=ExcludeFromCodeCoverage /p:AltCoverVerbosity=Warning’

Arguments string explained:

  1. /p:AltCover=true (use AltCover package)
  2. /p:AltCoverForce=true (Use AltCover even if other packages are in place)
  3. /p:AltCoverReport=$(Build.SourcesDirectory)/coverage.$(ProjectName).opencover.xml (OutPut test project in opencover.xml)
  4. /p:AltCoverAssemblyExcludeFilter=Tests (exclude filter: all tests projects)
  5. /p:AltCoverAssemblyFilter=Tests|?( nameofproject1|| nameofproject2) (but include these)
  6. /p:AltCoverAttributeFilter=ExcludeFromCodeCoverage (apply exclude filter)
  7. /p:AltCoverVerbosity=Warning (log level)
Code Coverage with AltCover: How to test.Net Code in Azure DevOps YAML pipelines (9)

Result = transform OpenCover File into Cobertura

Code Coverage with AltCover: How to test.Net Code in Azure DevOps YAML pipelines (10)
Code Coverage with AltCover: How to test.Net Code in Azure DevOps YAML pipelines (11)
Code Coverage with AltCover: How to test.Net Code in Azure DevOps YAML pipelines (12)

Cobertura is a reporting tool. It helps developers and teams assess the quality of their code by visualising test coverage and identifying areas that require additional testing.

Its open-source nature and integration capabilities make it a valuable asset in the development and testing process.

it is also called a parser because it takes data and parses it into a nice report.

Code Coverage with AltCover: How to test.Net Code in Azure DevOps YAML pipelines (13)
Code Coverage with AltCover: How to test.Net Code in Azure DevOps YAML pipelines (14)
Code Coverage with AltCover: How to test.Net Code in Azure DevOps YAML pipelines (15)

Alt Cover Official documentation

Medium

Code Coverage with AltCover: How to test .Net Code in Azure DevOps YAML pipelines (2024)
Top Articles
Latest Posts
Article information

Author: Msgr. Refugio Daniel

Last Updated:

Views: 6588

Rating: 4.3 / 5 (54 voted)

Reviews: 93% of readers found this page helpful

Author information

Name: Msgr. Refugio Daniel

Birthday: 1999-09-15

Address: 8416 Beatty Center, Derekfort, VA 72092-0500

Phone: +6838967160603

Job: Mining Executive

Hobby: Woodworking, Knitting, Fishing, Coffee roasting, Kayaking, Horseback riding, Kite flying

Introduction: My name is Msgr. Refugio Daniel, I am a fine, precious, encouraging, calm, glamorous, vivacious, friendly person who loves writing and wants to share my knowledge and understanding with you.