diff --git a/config/_default/menus/main.en.yaml b/config/_default/menus/main.en.yaml index 717ed0196c41e..b173cdb562a6b 100644 --- a/config/_default/menus/main.en.yaml +++ b/config/_default/menus/main.en.yaml @@ -5133,6 +5133,11 @@ menu: parent: code_coverage identifier: code_coverage_setup weight: 1 + - name: Data Collected + url: code_coverage/data_collected/ + parent: code_coverage + identifier: code_coverage_data_collected + weight: 2 - name: Quality Gates url: quality_gates/ pre: ci diff --git a/content/en/code_coverage/_index.md b/content/en/code_coverage/_index.md index 0866765293648..6020ff1341729 100644 --- a/content/en/code_coverage/_index.md +++ b/content/en/code_coverage/_index.md @@ -4,6 +4,9 @@ further_reading: - link: "/code_coverage/setup" tag: "Documentation" text: "Set up Code Coverage" + - link: "/code_coverage/data_collected" + tag: "Documentation" + text: "Learn what data is collected for Code Coverage" algolia: tags: ["code coverage"] --- diff --git a/content/en/code_coverage/data_collected.md b/content/en/code_coverage/data_collected.md new file mode 100644 index 0000000000000..ac825500f592b --- /dev/null +++ b/content/en/code_coverage/data_collected.md @@ -0,0 +1,56 @@ +--- +title: Code Coverage Data Collected +further_reading: + - link: "/code_coverage" + tag: "Documentation" + text: "Code Coverage" + - link: "/code_coverage/setup" + tag: "Documentation" + text: "Set up Code Coverage" +--- + +{{< callout url="http://datadoghq.com/product-preview/code-coverage/" >}} +Code Coverage is in Preview. This product replaces Test Optimization's code coverage feature, which is being deprecated. Complete the form to request access for the new Code Coverage product. +{{< /callout >}} + +## Source Code Provider Integration + +The exact data received by Datadog depends on your source code provider type: + +{{< tabs >}} +{{% tab "GitHub" %}} + +Code Coverage relies on the following GitHub webhooks: +* Pull request +* Pull request review +* Pull request review comment +* Push + +None of the webhooks include your source code content; they only include metadata about the pull request, such as title, description, author, labels, and commit SHAs. + +See GitHub's [webhook events and payloads documentation][1] for a detailed description of the data sent by these webhooks. + +[1]: https://docs.github.com/en/webhooks/webhook-events-and-payloads + +{{% /tab %}} +{{< /tabs >}} + +By default, when synchronizing your repositories, Datadog doesn’t store the actual content of files in your repository, only the Git commit and tree objects. + +See [Datadog Source Code Integration][1] for more information about how Datadog integrates with your source code provider. + +## Code Coverage Report Upload + +The `datadog-ci coverage upload` command sends the following data to Datadog: +- **Coverage reports**: The report files, which contain the coverage data for your codebase. The data depends on the coverage tool and report format you are using, and normally includes file paths, line numbers, and coverage percentages. +- **Git metadata**: Git repository URL, branch name, commit SHA, timestamp, author information, and list of file paths that were changed in the commit. You can disable Git metadata upload by adding `--skip-git-metadata-upload=1` to the command. +- **Git diff summary**: List of file paths that were changed in the commit, along with the numbers of added and removed lines. You can disable Diff data upload by adding `--upload-git-diff=0` to the command. +- **CI metadata**: Information about the CI environment, such as the CI provider, job ID, and pipeline ID. + +No source code is uploaded to Datadog. + +## Further reading + +{{< partial name="whats-next/whats-next.html" >}} + +[1]: /integrations/guide/source-code-integration diff --git a/content/en/code_coverage/setup.md b/content/en/code_coverage/setup.md index 7a31518f23395..445a9a303e42c 100644 --- a/content/en/code_coverage/setup.md +++ b/content/en/code_coverage/setup.md @@ -4,6 +4,9 @@ further_reading: - link: "/code_coverage" tag: "Documentation" text: "Code Coverage" + - link: "/code_coverage/data_collected" + tag: "Documentation" + text: "Learn what data is collected for Code Coverage" --- {{< callout url="http://datadoghq.com/product-preview/code-coverage/" >}} @@ -41,28 +44,36 @@ The following webhooks are required: | Pull request review comment | Receive PR data updates. | | Push | Receive Git commit metadata. | +If everything is configured correctly, a green check mark is displayed in Datadog's [GitHub Integration][2] page: +{{< img src="/code_coverage/github_app_success.png" alt="GitHub App integration success check" style="width:100%" >}} +
If you have a Datadog-managed Marketplace App or a custom app with default settings, the required permissions and webhooks are included.
[1]: /integrations/github/#github-apps-1 +[2]: https://app.datadoghq.com/integrations/github/configuration {{% /tab %}} {{< /tabs >}} +See [Data Collected][1] for details on what data is collected from your source code provider. + ## Data access permissions -If you are using [custom roles][1] rather than [Datadog-managed roles][2], be sure to enable the `Code Coverage Read` permission for the roles that need to view code coverage data. +If you are using [custom roles][2] rather than [Datadog-managed roles][3], be sure to enable the `Code Coverage Read` permission for the roles that need to view code coverage data. -Navigate to [Roles settings][3], click `Edit` on the role you need, add the `Code Coverage Read` permission to the role, and save the changes. +Navigate to [Roles settings][4], click `Edit` on the role you need, add the `Code Coverage Read` permission to the role, and save the changes. ## PR Gates If you wish to gate on PR coverage, configure PR Gates rules in Datadog. -Navigate to [PR Gates rule creation][4] and configure a rule to gate on total or patch coverage. +Navigate to [PR Gates rule creation][5] and configure a rule to gate on total or patch coverage. ## Upload code coverage reports Update your CI pipeline to upload code coverage report files to Datadog. This involves installing and running the `datadog-ci` CLI in your CI environment. +See [Data Collected][6] for details on what data is collected during code coverage report upload. + ### Supported coverage report formats Datadog supports the following coverage data formats—expand for examples: @@ -238,7 +249,7 @@ end_of_record ### Install the datadog-ci CLI -Install the [`datadog-ci`][5] CLI globally using `npm`: +Install the [`datadog-ci`][7] CLI globally using `npm`: {{< code-block lang="shell" >}} npm install -g @datadog/datadog-ci @@ -246,7 +257,7 @@ npm install -g @datadog/datadog-ci #### Standalone binary -If installing Node.js in the CI is an issue, standalone binaries are provided with [Datadog CI releases][6]. Only _linux-x64_, _linux-arm64_, _darwin-x64_, _darwin-arm64_ (MacOS) and _win-x64_ (Windows) are supported. To install, run the following from your terminal: +If installing Node.js in the CI is an issue, standalone binaries are provided with [Datadog CI releases][8]. Only _linux-x64_, _linux-arm64_, _darwin-x64_, _darwin-arm64_ (MacOS) and _win-x64_ (Windows) are supported. To install, run the following from your terminal: {{< tabs >}} {{% tab "Linux" %}} @@ -285,7 +296,7 @@ Start-Process -FilePath "./datadog-ci.exe" -ArgumentList version ### Uploading coverage reports -To upload your code coverage reports to Datadog, run the following command. Provide a valid [Datadog API key][7] (`DD_API_KEY`), and one or more file paths to either the coverage report files directly or directories containing them: +To upload your code coverage reports to Datadog, run the following command. Provide a valid [Datadog API key][9] (`DD_API_KEY`), and one or more file paths to either the coverage report files directly or directories containing them: {{< tabs >}} {{% tab "GitHub Actions" %}} @@ -299,15 +310,14 @@ steps: DD_SITE: {{< region-param key="dd_site" >}} -[1]: https://docs.github.com/en/actions/learn-github-actions/expressions#always {{% /tab %}} {{< /tabs >}} The command recursively searches the specified directories for supported coverage report files, so specifying the current directory (`.`) is usually sufficient. -See the [`datadog-ci` documentation][8] for more details on the `datadog-ci coverage upload` command. +See the [`datadog-ci` documentation][10] for more details on the `datadog-ci coverage upload` command. Shortly after the code coverage report upload is finished, Datadog adds a PR comment with code coverage percentage values. -You can also view your coverage data aggregated by pull request in the [Code Coverage page][9] in Datadog, with the ability to examine individual files and lines of code. +You can also view your coverage data aggregated by pull request in the [Code Coverage page][11] in Datadog, with the ability to examine individual files and lines of code. {{< img src="/code_coverage/pr_details.png" text="Code Coverage PR details page in Datadog" style="width:100%" >}} @@ -315,12 +325,14 @@ You can also view your coverage data aggregated by pull request in the [Code Cov {{< partial name="whats-next/whats-next.html" >}} -[1]: /account_management/rbac/permissions/#custom-roles -[2]: /account_management/rbac/permissions/#managed-roles -[3]: https://app.datadoghq.com/organization-settings/roles -[4]: https://app.datadoghq.com/ci/pr-gates/rule/create -[5]: https://www.npmjs.com/package/@datadog/datadog-ci -[6]: https://github.com/DataDog/datadog-ci/releases -[7]: https://app.datadoghq.com/organization-settings/api-keys -[8]: https://github.com/DataDog/datadog-ci/blob/master/src/commands/coverage/README.md -[9]: https://app.datadoghq.com/ci/code-coverage +[1]: /code_coverage/data_collected/#source-code-provider-integration +[2]: /account_management/rbac/permissions/#custom-roles +[3]: /account_management/rbac/permissions/#managed-roles +[4]: https://app.datadoghq.com/organization-settings/roles +[5]: https://app.datadoghq.com/ci/pr-gates/rule/create +[6]: /code_coverage/data_collected/#code-coverage-report-upload +[7]: https://www.npmjs.com/package/@datadog/datadog-ci +[8]: https://github.com/DataDog/datadog-ci/releases +[9]: https://app.datadoghq.com/organization-settings/api-keys +[10]: https://github.com/DataDog/datadog-ci/blob/master/src/commands/coverage/README.md +[11]: https://app.datadoghq.com/ci/code-coverage diff --git a/static/images/code_coverage/github_app_success.png b/static/images/code_coverage/github_app_success.png new file mode 100644 index 0000000000000..b729e4e5fa8c5 Binary files /dev/null and b/static/images/code_coverage/github_app_success.png differ