Union code coverage across all six pipelines by aaronburtle · Pull Request #3716 · Azure/data-api-builder

aaronburtle · 2026-07-09T20:51:54Z

Why make this change?

What is this change?

Adds a best-effort Total Code Coverage job to the existing unit-test
pipeline (unittest-pipelines.yml) that surfaces the true combined coverage
across all six test pipelines, instead of the misleadingly low per-pipeline
numbers.
- DAB's tests are split across six independent ADO pipelines (unit + MsSql,
  PostgreSql, MySql, DwSql, CosmosDb). Each publishes its own coverage tab and
  nothing merges them, so every pipeline in isolation reads much lower than the union.
How it works:
- The build job publishes this run's unit cobertura as a coverage-unit
  pipeline artifact.
- The new total_code_coverage job (dependsOn: build) downloads that plus the
  latest available cobertura from the five DB pipelines
  (DownloadPipelineArtifact@2, definition IDs DwSql=1, MsSql=2, MySql=3,
  PgSql=4, Cosmos=6).
- scripts/merge-coverage.ps1 unions all reports and emits one combined
  Cobertura.xml, which PublishCodeCoverageResults@2 publishes.
Design notes:
- Rolling dashboard, not a synchronized per-PR gate. DB coverage is pulled
  with runVersion: latest, so it may reflect an older commit ADO offers no
  primitive to block on all six sibling runs from inside one of them. A true
  per-PR guarantee would require consolidating all suites into one pipeline with
  a dependsOn-all aggregation job. This is not necessary for tracking code coverage.
- Best-effort: continueOnError on the job and every step, so it can never
  fail or block unit CI. If aggregation fails, the build job's own unit-only
  coverage remains as the tab fallback.
- The merge keys on fully-qualified class name + line (agent-path independent)
  so points union correctly across Windows/Linux agents, and excludes *.Tests
  assemblies.

How was this tested?

Integration Tests
Unit Tests

Validated scripts/merge-coverage.ps1 locally against real cobertura from all six
pipelines (combined = 76.77% branch / 83.33% line) and against synthetic reports
covering cross-agent path union, branch-condition max-merge, test-assembly
exclusion, and well-formed Cobertura output. Pipeline changes are CI-only.

Sample Request(s)

N/A — CI/pipeline change only; no REST, GraphQL, or CLI surface is affected.

Local usage of the merge script:

./scripts/merge-coverage.ps1 -Path <folder-of-cobertura> -OutFile combined/Cobertura.xml

…standalone

…gate

…pipeline IDs

…osmos=6)

Copilot

Pull request overview

This PR adds a “Total Code Coverage” aggregation mechanism to provide a more accurate combined (union) coverage number across DAB’s six independent Azure DevOps pipelines by downloading each pipeline’s Cobertura output and merging them into a single report.

Changes:

Introduces scripts/merge-coverage.ps1, a dependency-free Cobertura union merger that can also emit a merged Cobertura XML for publishing.
Updates the unit-test pipeline to publish its coverage as a pipeline artifact and adds a new best-effort job that downloads latest DB-pipeline coverage and publishes a combined coverage report.
Extends the shared build/unit-test template to stage and publish the unit coverage artifact used by the aggregation job.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
scripts/merge-coverage.ps1	Adds a PowerShell Cobertura union-merger (line + branch) and optional merged XML emitter.
.pipelines/unittest-pipelines.yml	Converts the unit pipeline to multi-job and adds a best-effort “Total Code Coverage” aggregation job.
.pipelines/templates/build-pipelines.yml	Publishes unit Cobertura files as a pipeline artifact (`coverage-unit`) for later aggregation.

…ad by name for aggregation

…or, warn on absent sources, map pipeline IDs

…r-edge branch union with floor fallback; guard 0/0 and de-dup inputs

…eport below either bound

…readability

souvikghosh04

LGTM

souvikghosh04 · 2026-07-15T15:31:54Z

+# Publish the RAW cobertura as a named pipeline artifact so the unit pipeline's
+# 'Total Code Coverage' job can download and union it. (PublishCodeCoverageResults@1
+# only exposes an HTML report artifact, not the raw XML, so aggregation needs this.)
+- task: CopyFiles@2
+  displayName: 'Stage raw coverage for aggregation'
+  inputs:
+    sourceFolder: '$(Agent.TempDirectory)'
+    contents: '**/*.cobertura.xml'
+    targetFolder: '$(Build.ArtifactStagingDirectory)/coverage'
+- task: PublishPipelineArtifact@1
+  displayName: 'Publish coverage artifact for aggregation'
+  inputs:
+    targetPath: '$(Build.ArtifactStagingDirectory)/coverage'
+    artifact: 'coverage-cosmos'


could this be moved to a separate template file? seems these are duplicated in all the pipeline yamls.

souvikghosh04 · 2026-07-15T15:34:31Z

+  - task: DownloadPipelineArtifact@2
+    displayName: 'Download MsSql coverage (latest)'
+    continueOnError: true
+    inputs:
+      source: 'specific'
+      project: '$(System.TeamProject)'
+      pipeline: '2'
+      runVersion: 'latest'
+      patterns: '**/*.cobertura.xml'
+      path: '$(Pipeline.Workspace)/db/mssql'
+  - task: DownloadPipelineArtifact@2
+    displayName: 'Download PostgreSql coverage (latest)'
+    continueOnError: true
+    inputs:
+      source: 'specific'
+      project: '$(System.TeamProject)'
+      pipeline: '4'
+      runVersion: 'latest'
+      patterns: '**/*.cobertura.xml'
+      path: '$(Pipeline.Workspace)/db/pg'
+  - task: DownloadPipelineArtifact@2
+    displayName: 'Download MySql coverage (latest)'
+    continueOnError: true
+    inputs:
+      source: 'specific'
+      project: '$(System.TeamProject)'
+      pipeline: '3'
+      runVersion: 'latest'
+      patterns: '**/*.cobertura.xml'
+      path: '$(Pipeline.Workspace)/db/mysql'
+  - task: DownloadPipelineArtifact@2
+    displayName: 'Download DwSql coverage (latest)'
+    continueOnError: true
+    inputs:
+      source: 'specific'
+      project: '$(System.TeamProject)'
+      pipeline: '1'
+      runVersion: 'latest'
+      patterns: '**/*.cobertura.xml'
+      path: '$(Pipeline.Workspace)/db/dwsql'
+  - task: DownloadPipelineArtifact@2
+    displayName: 'Download CosmosDb coverage (latest)'
+    continueOnError: true
+    inputs:
+      source: 'specific'
+      project: '$(System.TeamProject)'
+      pipeline: '6'
+      runVersion: 'latest'
+      patterns: '**/*.cobertura.xml'
+      path: '$(Pipeline.Workspace)/db/cosmos'


DownloadPipelineArtifact with no artifact: pulls every artifact of the run.. means all of the referenced run's artifacts are considered. The patterns: '**/.cobertura.xml' filter limits which files are downloaded, but this can still be slower/heavier than targeting the specifically-named coverage- artifacts you now publish. Since you control the artifact names, consider filtering to them to reduce download volume

aaronburtle added 8 commits July 9, 2026 12:08

Add combined coverage merge script + draft aggregator pipeline

e43e66e

Aggregator reuses existing coverage artifacts (no pipeline changes)

4633c65

Name run, fill best-guess pipeline sources for Total Code Coverage

abe2d34

Document one-time ADO registration + run-once trigger arming

4f2c030

Integrate Total Code Coverage aggregation into unit pipeline; remove …

6edd503

…standalone

Clarify combined coverage is a rolling dashboard, not a synchronized …

742b328

…gate

Drop ReportGenerator; merge script emits combined Cobertura; fill DB …

954bfb4

…pipeline IDs

Correct DB pipeline ID mapping (DwSql=1, MsSql=2, MySql=3, PgSql=4, C…

f1986f1

…osmos=6)

Copilot AI review requested due to automatic review settings July 9, 2026 20:51

aaronburtle requested review from Alekhya-Polavarapu, Aniruddh25, JerryNixon, RubenCerna2079, anushakolan, rusamant, sourabh1007, souvikghosh04, stuartpa and vadeveka as code owners July 9, 2026 20:51

Copilot started reviewing on behalf of aaronburtle July 9, 2026 20:52 View session

Copilot AI reviewed Jul 9, 2026

View reviewed changes

Comment thread .pipelines/unittest-pipelines.yml

Publish raw cobertura as named artifact from each DB pipeline; downlo…

07b9f4b

…ad by name for aggregation

aaronburtle self-assigned this Jul 9, 2026

aaronburtle added testing Test related issue. assign-for-review labels Jul 9, 2026

aaronburtle added this to Data API builder Jul 9, 2026

github-project-automation Bot moved this to Todo in Data API builder Jul 9, 2026

aaronburtle added this to the July 2026 milestone Jul 9, 2026

github-actions Bot assigned anushakolan and RubenCerna2079 Jul 9, 2026

github-actions Bot removed the assign-for-review label Jul 9, 2026

aaronburtle unassigned anushakolan Jul 9, 2026

aaronburtle added the assign-for-review label Jul 9, 2026

github-actions Bot assigned souvikghosh04 Jul 9, 2026

github-actions Bot removed the assign-for-review label Jul 9, 2026

aaronburtle added 4 commits July 9, 2026 16:57

Trim Total Code Coverage comment

f4e07bb

Address review: capture Windows-job coverage, document branch% as flo…

da9dd77

…or, warn on absent sources, map pipeline IDs

Fix mssql template regression (restore received-files copy); exact pe…

147a0d0

…r-edge branch union with floor fallback; guard 0/0 and de-dup inputs

Take max of per-edge union and aggregate max so mixed sources never r…

1e662f6

…eport below either bound

RubenCerna2079 requested changes Jul 13, 2026

View reviewed changes

github-project-automation Bot moved this from Todo to Review In Progress in Data API builder Jul 13, 2026

aaronburtle and others added 2 commits July 14, 2026 13:05

Address Ruben review nits: wording tweaks and multi-line if/else for …

0984ed8

…readability

Merge branch 'main' into dev/aaronburtle/combined-coverage-union

cc35f22

souvikghosh04 approved these changes Jul 15, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Union code coverage across all six pipelines#3716

Union code coverage across all six pipelines#3716
aaronburtle wants to merge 15 commits into
mainfrom
dev/aaronburtle/combined-coverage-union

aaronburtle commented Jul 9, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

souvikghosh04 left a comment

Uh oh!

souvikghosh04 Jul 15, 2026

Uh oh!

souvikghosh04 Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

aaronburtle commented Jul 9, 2026

Why make this change?

What is this change?

How was this tested?

Sample Request(s)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

souvikghosh04 left a comment

Choose a reason for hiding this comment

Uh oh!

souvikghosh04 Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

souvikghosh04 Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants