Code coverage results are waiting until timeout

[Apoorva38]

Extension: BuildQualityChecks
Environment: Azure DevOps Services (cloud)
Agent type: Microsoft-Self-hosted (linux Agent)
Pipeline type: yaml pipeline

below task has suddenly stopped working in multiple pipelines:

• task: BuildQualityChecks@9
  displayName: Code Coverage Check
  inputs:
  checkCoverage: true
  coverageFailOption: 'fixed'
  coverageType: 'lines'
  coverageThreshold: ${{ parameters.coverageThreshold }}

Error :

Waiting for code coverage data...
Waiting for code coverage data...

[ReneSchumacher]

Hi @Apoorva38,

I believe I have already responded to your email to our support address. If not, let me know and I will respond here with more details.

[PolinaRuban]

Can you please make the answer and/or solution public? We are also seeing the issue like that

Yeah we started seeing this again this week. #237

We had just rolled back our workaround referenced in #236 because it created a disk space issue with a mono nuget repo. Rolled back, enjoyed a few days of bliss and now the timeout is back. lol.

[ReneSchumacher]

Hi everyone,

sorry for the late response here, we're still trying to investigate the issue. Simply put, there is nothing we can do from the task side to solve it. Build Quality Checks currently relies on the Azure DevOps APIs to return the proper coverage values after they have been published to Azure DevOps. In some cases, though, it seems that Azure DevOps fails to return the data quickly and, thus, BQC runs into a timeout.

Sometimes, increasing the timeout by setting the PSGer.Cover.MaxWaitTime variable to a high value in milliseconds (default is 600,000 = 10 minutes) can help, but it will dramatically slow down your pipeline. We are working with the Azure DevOps product group to find the root cause for this, but the BQC team is not part of Azure DevOps and cannot really investigate the root cause.

To get rid of the API dependency, we are working on a new version of BQC that will take care of parsing the various coverage files itself, so it cannot be affected by upload or processing issues anymore. If you see timeouts in BQC, please do the following:

1. Run your pipeline again with the variables System.Debug and BQC.LogRawData set to true.
2. Search the BQC log for the line Waiting for code coverage data.... There should be a section of ~10 minutes of log output that shows this message and - when detailed logging has been properly activated - some raw API response that looks similar to this:
   {"coverageData":[],"build":{"id":"{build id}","url":https://dev.azure.com/{your org}/_apis/build/Builds/{build id},"deltaBuild":null,"coverageDetailedSummaryStatus":"codeCoverageSuccess"}
3. Check if all API outputs look the same (very likely) and make sure that there is no status property in the response and the coverageData element is always an empty array.
4. Check if coverage values are displayed on your build summary page after the build has finished. If you don't see coverage values there, there might be a problem with your coverage data creation or the configuration of the task that should upload coverage data to Azure DevOps.

If you see the data described in step 3, please open a support ticket with Azure DevOps and send the debug logs of your test step (e.g., Visual Studio Test, .NET Core CLI) and the BQC task with the support ticket. You can also send the logs and case number to PSGerExtSupport@microsoft.com, so we can directly work with our support engineers if needed.

If, however, you see coverage data or a status property in the API result and BQC still keeps timing out, open an issue here and send your logs to PSGerExtSupport@microsoft.com.