Skip to content

DAC for LLM Observability #30771

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Aug 4, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions config/_default/menus/main.en.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4713,16 +4713,21 @@ menu:
parent: llm_obs
identifier: llm_obs_experiments
weight: 5
- name: Data Security and RBAC
url: llm_observability/data_security_and_rbac
parent: llm_obs
identifier: llm_obs_data_security_and_rbac
weight: 6
- name: Terms and Concepts
url: llm_observability/terms/
parent: llm_obs
identifier: llm_obs_terms
weight: 6
weight: 7
- name: Guides
url: llm_observability/guide/
parent: llm_obs
identifier: llm_obs_guide
weight: 7
weight: 8
- name: CI Visibility
url: continuous_integration/
pre: ci
Expand Down
24 changes: 13 additions & 11 deletions content/en/account_management/rbac/data_access.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,10 @@ Log in as a user assigned the Datadog Admin role, or any user with a role in you

In order to create a Restricted Dataset, identify the data to be restricted with a query.

{{< img src="/account_management/rbac/restricted_dataset.png" alt="Create a Restricted Dataset dialog. Selects data in RUM, APM, Logs, and Metrics matching the tag service:hr. Grants access to a Privileged access team.">}}
{{< img src="/account_management/rbac/restricted_dataset-2.png" alt="Create a Restricted Dataset dialog. Selects data in RUM, APM, Logs, and Metrics matching the tag service:hr. Grants access to a Privileged access team.">}}

Name Dataset
: A descriptive name to help users understand what data is contained in the dataset.
: A descriptive name to help users understand what data is contained in the dataset.

Select data to be included in this Dataset
: The boundary definition that describes which data to restrict to a specific set of users. Boundaries are query statements with limitations that allow an access manager to define the scope of sensitive data to be protected. The [supported telemetry types][10] are custom metrics, RUM sessions, APM traces, logs, cloud costs, error tracking issues, and CI Visibility pipelines.
Expand All @@ -61,7 +61,7 @@ After completing all the fields to define the dataset, click **Create Restricted
You may create a maximum of 100 Restricted Datasets. If you need a higher limit, reach out to Support.

### API
The Data Access Control API is under development and should be considered unstable. Future versions may be backward incompatible.
The Data Access Control API is under development and should be considered unstable. Future versions may be backward incompatible.

Terraform support will be announced after Data Access Control is generally available.

Expand All @@ -75,6 +75,7 @@ Terraform support will be announced after Data Access Control is generally avail
- Error Tracking issues
- Logs
- RUM sessions
- LLM Observability

## Usage constraints

Expand All @@ -92,6 +93,7 @@ Playlists are collections of Session Replays you can aggregate in a folder-like
### Logs
Data Access Control is separate from the existing [Logs RBAC permissions][11] feature, also known as log restriction queries. To use Data Access Control with Log Management, first request access to Data Access Control. Next, manually migrate your configuration from Log Management permissions to Data Access Control.


## Select tags for access

Each Restricted Dataset can control access to multiple types of data, such as metrics. You are free to use the same or different tags across multiple types of telemetry. Within each telemetry type, you must use a _single_ tag or attribute to define your access strategy.
Expand All @@ -101,7 +103,7 @@ If you have too many combinations of tags or attributes to fit within these cons
### Supported example

#### Restricted Dataset 1
- Telemetry Type: RUM
- Telemetry Type: RUM
- Filters: `@application.id:ABCD`

#### Restricted Dataset 2
Expand All @@ -112,23 +114,23 @@ If you have too many combinations of tags or attributes to fit within these cons

### Not supported example

#### Restricted Dataset 1:
* Telemetry type: RUM
#### Restricted Dataset 1:
* Telemetry type: RUM
* Filters: `@application.id:ABCD`

#### Restricted Dataset 2:
* Telemetry type: RUM
* Telemetry type: RUM
* Filters: `env:prod`

Restricted Dataset 1 uses `@application.id` as the tag for RUM data, so a new Restricted Dataset can't change to a different tag. Instead, consider reconfiguring Restricted Dataset 2 to use `@application.id`, or changing all of your Restricted Datasets with RUM data to use another tag.

### Not supported example

#### Restricted Dataset 1:
* Telemetry type: RUM
#### Restricted Dataset 1:
* Telemetry type: RUM
* Filters: `@application.id:ABCD`

#### Restricted Dataset 2:
#### Restricted Dataset 2:
* Telemetry type: RUM
* Filters: `@application.id:IJKL` `env:prod`

Expand All @@ -150,7 +152,7 @@ For example, if you have a single application that is instrumented with Real Use
* Telemetry type: RUM
* Filters: `@application.id:<rum-app-id>`
* **Grant access:**
* Teams or roles of users who can see this RUM data
* Teams or roles of users who can see this RUM data

This configuration example would protect the RUM data from this application, and keep other data from this application available to existing users in your organization.

Expand Down
23 changes: 23 additions & 0 deletions content/en/llm_observability/data_security_and_rbac.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
title: Data Security and RBAC
further_reading:
- link: "/account_management/rbac/data_access"
tag: "Documentation"
text: "Learn more about data access controls"
---
{{< whatsnext desc=" ">}}
{{< nextlink href="https://datadoghq.com/legal/hipaa-eligible-services">}}<u>HIPAA-Eligible Services</u>: Datadog Legal's list of HIPAA-eligible services{{< /nextlink >}}
{{< nextlink href="/llm_observability/evaluations/#sensitive-data-scanner-integration">}}<u>Sensitive Data Scanning for LLM Observability</u>: Redact sensitive information in your LLM application{{< /nextlink >}}
{{< /whatsnext >}}

## Data Access Control

LLM Observability allows you to restrict access to potentially sensitive data associated with your ML applications to only certain teams and roles in your organization. This is particularly important when your LLM applications process sensitive information such as personal data, proprietary business information, or confidential user interactions.

Access controls in LLM Observability are built on Datadog's [Data Access Control][1] feature, which enables enables you to regulate access to data deemed sensitive. You can use the `ml_app` tag to identify and restrict access to specific LLM applications within your organization.

## Further reading

{{< partial name="whats-next/whats-next.html" >}}

[1]: /account_management/rbac/data_access
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading