-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Cloud Run Jobs Docs #32373
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
nhulston
wants to merge
14
commits into
master
Choose a base branch
from
nicholas.hulston/cloud-run-jobs
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+568
−31
Open
Cloud Run Jobs Docs #32373
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
80f167b
cloud run jobs homepage
nhulston 2fb7144
runtimes skeleton
nhulston f92d0f3
(chore): update dogstatsd code example link to point directly to dist…
nhulston 3af2cca
Implement Python
nhulston ce55ef5
(chore): use single quotes in node.js example code
nhulston 08d061b
Implement Node.js
nhulston 136103d
Implement Go
nhulston a3644d4
Implement Java
nhulston 2a84ff6
Implement .NET
nhulston ee2461a
Merge branch 'master' into nicholas.hulston/cloud-run-jobs
nhulston cbbd7f1
clarify distinction between Alpine and ManyLinux
nhulston 51dc502
rename ManyLinux to Standard Linux
nhulston 38871cb
(chore) remove extra space
nhulston 56e12c3
replacing preview callout
cswatt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| --- | ||
| title: Instrumenting Cloud Run Jobs | ||
| type: multi-code-lang | ||
|
|
||
| --- | ||
|
|
||
| {{< callout url="https://forms.gle/DNbZDFb3mDTt2vxb7" | ||
| btn_hidden="false" header="Join the Preview!">}} | ||
| Serverless Monitoring for Google Cloud Run Jobs is in Preview. Use this form to submit your request today. | ||
| {{< /callout >}} | ||
|
|
||
| First, set up the [Datadog-Google Cloud Platform integration][1] to collect metrics and logs from Google Cloud services. Remember to add the `cloud asset viewer` role to your service account and enable the Cloud Asset Inventory API in Google Cloud. | ||
|
|
||
| Then, select your runtime below for instructions on how to instrument your application: | ||
|
|
||
| {{% container-languages path="google_cloud_run/jobs" jobs="true" %}} | ||
|
|
||
| [1]: /integrations/google-cloud-platform/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| --- | ||
| title: Instrumenting a .NET Cloud Run Job | ||
| code_lang: dotnet | ||
| type: multi-code-lang | ||
| code_lang_weight: 50 | ||
| further_reading: | ||
| - link: '/tracing/trace_collection/automatic_instrumentation/dd_libraries/dotnet-core/?tab=linux' | ||
| tag: 'Documentation' | ||
| text: 'Tracing .NET Core Applications' | ||
| - link: '/tracing/other_telemetry/connect_logs_and_traces/dotnet/' | ||
| tag: 'Documentation' | ||
| text: 'Correlating .NET Logs and Traces' | ||
| --- | ||
|
|
||
| ## Setup | ||
|
|
||
| <div class="alert alert-info">A sample application is <a href="https://github.com/DataDog/serverless-gcp-sample-apps/tree/main/cloud-run/in-container/dotnet">available on GitHub</a>.</div> | ||
| <div class="alert alert-info"> | ||
| For full visibility and access to all Datadog features in Cloud Run Jobs, | ||
| ensure you’ve <a href="http://localhost:1313/integrations/google_cloud_platform/">installed the Google Cloud integration</a> | ||
| and are using <a href="https://hub.docker.com/r/datadog/serverless-init#180">serverless-init version 1.8.0 or later</a>. | ||
| </div> | ||
|
|
||
| 1. **Install the Datadog .NET tracer** in your Dockerfile. | ||
|
|
||
| Because GitHub requests are rate limited, you must pass a GitHub token saved in the environment variable `GITHUB_TOKEN` as a [Docker build secret][1] `--secret id=github-token,env=GITHUB_TOKEN`. | ||
|
|
||
| {{< tabs >}} | ||
| {{% tab "Standard Linux (glibc)" %}} | ||
| {{< code-block lang="dockerfile" filename="Dockerfile" disable_copy="false" collapsible="true" >}} | ||
| RUN --mount=type=secret,id=github-token,env=GITHUB_TOKEN \ | ||
| chmod +x /app/dotnet.sh && /app/dotnet.sh | ||
| {{< /code-block >}} | ||
| {{% /tab %}} | ||
|
|
||
| {{% tab "Alpine (musl)" %}} | ||
| {{< code-block lang="dockerfile" filename="Dockerfile" disable_copy="false" collapsible="true" >}} | ||
| # For alpine use datadog-dotnet-apm-2.57.0-musl.tar.gz | ||
| ARG TRACER_VERSION | ||
| ADD https://github.com/DataDog/dd-trace-dotnet/releases/download/v${TRACER_VERSION}/datadog-dotnet-apm-${TRACER_VERSION}.tar.gz /tmp/datadog-dotnet-apm.tar.gz | ||
|
|
||
| RUN mkdir -p /dd_tracer/dotnet/ && tar -xzvf /tmp/datadog-dotnet-apm.tar.gz -C /dd_tracer/dotnet/ && rm /tmp/datadog-dotnet-apm.tar.gz | ||
| {{< /code-block >}} | ||
| {{% /tab %}} | ||
| {{< /tabs >}} | ||
|
|
||
| **Note**: Cloud Run Jobs run to completion rather than serving requests, so auto instrumentation won't create a top-level "job" span. For end-to-end visibility, create your own root span. See the [.NET Custom Instrumentation][2] instructions. | ||
|
|
||
| For more information, see [Tracing .NET applications][3]. | ||
|
|
||
| 2. **Install serverless-init**. | ||
|
|
||
| {{% gcr-install-serverless-init cmd="\"dotnet\", \"dotnet.dll\"" %}} | ||
|
|
||
| 3. **Set up logs**. | ||
|
|
||
| To enable logging, set the environment variable `DD_LOGS_ENABLED=true`. This allows `serverless-init` to read logs from stdout and stderr. | ||
|
|
||
| Datadog also recommends setting the environment variables `DD_LOGS_INJECTION=true` and `DD_SOURCE=csharp` to enable advanced Datadog log parsing. | ||
|
|
||
| If you want multiline logs to be preserved in a single log message, Datadog recommends writing your logs in JSON format. For example, you can use a third-party logging library such as `Serilog`: | ||
|
|
||
| {{< code-block lang="csharp" disable_copy="false" >}} | ||
| using Serilog; | ||
|
|
||
| builder.Host.UseSerilog((context, config) => | ||
| { | ||
| config.WriteTo.Console(new Serilog.Formatting.Json.JsonFormatter(renderMessage: true)); | ||
| }); | ||
|
|
||
| logger.LogInformation("Hello World!"); | ||
| {{< /code-block >}} | ||
|
|
||
| For more information, see [Correlating .NET Logs and Traces][4]. | ||
|
|
||
| 4. **Configure your application**. | ||
|
|
||
| {{% gcr-configure %}} | ||
|
|
||
| 5. {{% gcr-service-label %}} | ||
|
|
||
| 6. **Send custom metrics**. | ||
|
|
||
| To send custom metrics, [install the DogStatsD client][5] and [view code examples][6]. In serverless, only the *distribution* metric type is supported. | ||
|
|
||
| {{% gcr-env-vars-in-container language="csharp" %}} | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| {{% gcr-troubleshooting %}} | ||
|
|
||
| ## Further reading | ||
|
|
||
| {{< partial name="whats-next/whats-next.html" >}} | ||
|
|
||
| [1]: https://docs.docker.com/build/building/secrets/ | ||
| [2]: /tracing/trace_collection/custom_instrumentation/dotnet/dd-api#instrument-methods-through-attributes | ||
| [3]: /tracing/trace_collection/automatic_instrumentation/dd_libraries/dotnet-core/?tab=linux | ||
| [4]: /tracing/other_telemetry/connect_logs_and_traces/dotnet/ | ||
| [5]: /developers/dogstatsd/?tab=dotnet#install-the-dogstatsd-client | ||
| [6]: /metrics/custom_metrics/dogstatsd_metrics_submission/?tab=dotnet#code-examples-5 |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why use the distribution metric example instead of count?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only distribution metrics are supported in Serverless
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reason has to do with the lack of
hostname, resulting in the backend thinking that unique metrics are actually duplicate retries of the same metric, resulting in lost & inaccurate metrics