[DOCS-11489] CCM new data source

content/en/cloud_cost_management/custom_allocation_rules.md

@@ -16,23 +16,24 @@
 
 The following allocation methods are available:
 
- | Allocation Method | Description | Use Case | Example |
- | ----------------  | ----------- | -------- | --------|
- | Even  | Split costs evenly among all destinations. | Scenarios where each team, project, or environment should be charged the same amount for a shared cost. | Untagged support costs are allocated evenly to teams `teamA`, `teamB`, and `teamC`. |
- | Custom  | Split costs to each destination based on percentages you define. | Scenarios where business rules or agreements dictate how much each team should pay. | Untagged support costs are allocated 60% to `teamA`, 30% to `teamB`, and 10% to `teamC`. |
- | Proportional by spend | Split costs based on each destination's share of total spend. | Scenarios where teams should pay in proportion to their actual spend. | Untagged support costs are allocated to teams `teamA`, `teamB`, and `teamC` based on their proportion of total spend on Amazon EC2.|
- | Dynamic by metric  | Split costs based each destination's share of total usage. | Scenarios where teams should pay in proportion to their actual usage. | Shared PostgreSQL costs are allocated by total query execution time per team. |
+ | Allocation Method | Description | Use Case |
+ | ----------------  | ----------- | -------- |
+ | Even  | Split costs evenly among all destinations. | Scenarios where each team, project, or environment should be charged the same amount for a shared cost. |
+ | Custom  | Split costs to each destination based on percentages you define. | Scenarios where business rules or agreements dictate how much each team should pay. |
+ | Proportional by spend | Split costs based on each destination's share of total spend. | Scenarios where teams should pay in proportion to their actual spend. |
+ | Dynamic by metric  | Split costs based on each destination's share of total usage. | Scenarios where teams should pay in proportion to their actual usage. |
 
-Custom allocation rules runs after [Tag Pipelines][1], enabling cost allocations on the latest user-defined tags. Costs are allocated on a daily basis. Cost allocations can be applied to AWS, Google Cloud, and Azure costs.
+Custom allocation rules run after [Tag Pipelines][1], enabling cost allocations on the latest user-defined tags. Costs are allocated on a daily basis. Cost allocations can be applied to AWS, Google Cloud, and Azure costs.
 
 ## Create a custom allocation rule
 
 ### Step 1 - Define the source
 
 1. Navigate to [Cloud Cost > Settings > Custom Allocation Rules][2] and click **Add New Rule** to start.
-2. From the dropdown, select the shared costs you want to allocate.
+1. Under **Define the source**, select the cost provider.
+1. Under **Define the costs to split**, select the shared costs you want to allocate.
 
-   _Example: Untagged support costs, shared database costs._
+_Example: Untagged support costs, shared database costs._
 
 ### Step 2 - Choose an allocation method
 
@@ -56,7 +57,7 @@
 
 With the custom percentage strategy, you can define static custom percentages for the destination tags you select. For example, if you have 3 destinations (`teamA`, `teamB`, `teamC`) you can allocate 60% to `teamA`, 30% to `teamB`, and 10% to `teamC`.
 
-{{< img src="cloud_cost/custom_allocation_rules/ui-custom.png" alt="The even split strategy as seen in Datadog" style="width:90%;" >}}
+{{< img src="cloud_cost/custom_allocation_rules/ui-custom-2.png" alt="The even split strategy as seen in Datadog" style="width:90%;" >}}
 
 {{% /tab %}}
 
@@ -70,15 +71,15 @@
 
 To create a rule for this allocation, you can:
 
-- Define the costs to allocate (source): **EC2 support fees** (`aws_product:support`).
+- Define the costs to allocate (source): **EC2 support fees** (`aws_product:support`). 
 - Choose the allocation method: **Proportional by spend**.
 - Choose the [destination tag](#step-3---define-the-destination) to split your costs by: **User** (`User A`, `User B`, `User C`).
 - Refine the allocation by applying [filters](#step-4---optional-apply-filters): **EC2** (`aws_product:ec2`).
-- Create suballocations by [partitioning](#step-5---optional-apply-a-partition) the allocation rule: **environment** (`env`).
+- Create suballocations by [partitioning](#step-4---optional-apply-a-partition) the allocation rule: **environment** (`env`).
 
 You can also specify how cost proportions should be partitioned to ensure segment-specific allocations. For example, if you partition your costs by environment using tags like `staging` and `production`, the proportions are calculated separately for each environment. This ensures allocations are based on the specific proportions within each partition.
 
-{{< img src="cloud_cost/custom_allocation_rules/ui-proportional-by-spend.png" alt="The proportional split strategy as seen in Datadog" style="width:90%;" >}}
+{{< img src="cloud_cost/custom_allocation_rules/ui-proportional-by-spend-2.png" alt="The proportional split strategy as seen in Datadog" style="width:90%;" >}}
 
 {{% /tab %}}
 
@@ -88,39 +89,89 @@
 
 Metrics-based allocation provides the ability to split up costs based on Datadog's [metrics queries][1]. By using performance metrics to allocate expenses, you can more accurately allocate costs based on application usage patterns.
 
-For example, this PostgreSQL metrics query `sum:postgresql.queries.time{*} by {user}.as_count()` tracks the total query execution time per user. The relative values are then used to determine what proportion of total PostgreSQL costs should be allocated to each user.
-
-To create a rule for this allocation, you could:
-
-- Define the costs to allocate (source): **PostGreSQL costs** (`azure_product_family:dbforpostgresql`).
-- Choose the allocation method: **Dynamic by metric**
-- Choose the [destination tag](#step-3---define-the-destination) to split your costs by: **User** (`User A`, `User B`, `User C`).
-- Refine the allocation by applying [filters](#step-4---optional-apply-filters): **EC2** (`aws_product:ec2`).
-- Define the metric query used to split the source costs: **Query execution time per user** (`sum:postgresql.queries.time{*}` by `{user}.as_count`).
-- Create suballocations by [partitioning](#step-5---optional-apply-a-partition) the allocation rule: **environment** (`env`).
-
-{{< img src="cloud_cost/custom_allocation_rules/ui-dynamic-by-metric.png" alt="The dynamic by metric split strategy as seen in Datadog" style="width:90%;" >}}
+For example, the Network query `sum:network.bytes_written[server_gateway_id:nat-*] by client_service and server_gateway_id` (shown below) tracks the total traffic volume through NAT gateways by service. The relative values are then used to determine what proportion of total NAT gateway costs should be allocated to each service.
+
+### Data source limitations
+
+Before creating this type of rule, be aware:
+
+- **Data sources support** - The dynamic by metric allocation method only supports specific data sources, listed in the dropdown below.
+
+{{% collapse-content title="List of supported data sources" level="h4" expanded=false id="supported-data-sources" %}}
+
+- APM Metrics
+- Application Security
+- Audit Trail
+- CI Pipelines
+- CI Tests
+- Cases
+- DORA Metrics
+- Database Queries
+- Deployment Gates
+- Events
+- Incidents
+- Kubernetes Troubleshooting
+- LLM Observability
+- Metrics
+- NetFlow
+- Network
+- Network Path
+- On-Call
+- Profiles
+- RUM
+- Recommendations
+- Security Signals
+- Software Composition Analysis
+- Synthetics CI Batches
+- Synthetics Runs
+- Usage
+- Workload Security
+- Workload Security Info
+
+{{% /collapse-content %}}
+
+- **Result limits** - Some data sources limit how many results they can show. The allocation rule automatically uses your highest-costing items first. For example, if your query editor shows a limit of 100 group bys but you have 101 results, the allocation rule distributes costs across the top 100 highest-costing group bys and ignores the remaining 1 completely. 
+- **Update frequency** - While CCM has 15 months of retention overall, individual data sources have different update patterns:
+  - Metrics data updates daily for the past 60 days
+  - Other data sources backfill 60 days at rule creation, then update daily for the past 7 days
+
+  If you need more frequent updates or longer retention, consider using the _Metrics_ data source.
+
+### Create a dynamic by metric allocation rule
+
+To create a rule for this allocation, you can, for example:
+
+- Define the source or costs to allocate (for example, **NAT gateway costs** (`aws_operation:NatGateway`)). 
+- Select the allocation method: **Dynamic by metric**
+  - Choose the data source: **Network**. Tip: Review available metrics and tags in the [Metrics Summary][2].
+- In the **"Define the metric to split the source costs"** section (which uses the same interface as the Metrics Explorer query editor):
+  - **Filters**: Apply filters to refine your metric query. For example, add `server_gateway_id:nat-*` to filter the metric to only return data for your NAT Gateway usage. These filters become the filter for your allocation.
+  - **Group bys**: Select group bys to define your destination and partition:
+    - **Destination**: The primary group by becomes your destination tag. For example, group by `client_service` to split costs by service.
+    - **Partition**: Additional group bys become your partition. For example, add another group by for `gateway_id` to create suballocations by Gateway ID.
+- In section 3, **"Choose the destination(s) to split costs across"**, specify which group by tag is your destination and which will be your partition. 
+{{< img src="cloud_cost/custom_allocation_rules/ui-dynamic-by-metric-4.png" alt="The dynamic by metric split strategy as seen in Datadog" style="width:90%;" >}}
 
 [1]: /metrics/#querying-metrics
+[2]: https://app.datadoghq.com/metric/summary
 
 {{% /tab %}}
 
 {{< /tabs >}}
 
-### Step 3 - Define the destination
+### Step 3 - Choose the destination(s) to split costs across
 
-Decide which dimensions, such as `team`, `department`, or `service`, receive the allocated costs. For example:
+1. Select the destinations you want to allocate costs to, such as `team`, `department`, or `service`, that receive the allocated costs.
 
-You can select multiple values for your destination tag. For instance, if you select the `team` tag, you can choose specific teams like `teamA`, `teamB`, and `teamC` to receive the allocated costs.
+   You can select multiple values for your destination tag. For instance, if you select the `team` tag, you can choose specific teams like `teamA`, `teamB`, and `teamC` to receive the allocated costs.
 
 ### Step 4 - (optional) Apply filter(s)
 
 Apply a filter across the entire allocation rule. Filters help you target the allocation rule to the relevant subset of your cloud spend.
 
-_Example: Only apply cost allocation where environment is production._
+- **Proportional by spend**: You can add filters to narrow the scope of your allocation. For example, filter by `aws_product:ec2` to create an allocation that only applies to EC2 costs, then proportionally distribute those costs based on each team's EC2 spend.
 
-- **Proportional by spend**: Let's say you allocate shared costs to the team tag, proportional to how much each team spends. You can add a filter, creating a cost allocation that is proportional to how much team spends on `aws_product` is `ec2`.
-- **Dynamic by metric**: Let's say you allocate shared PostgreSQL costs to the service tag, proportional to the query execution time of each service. You can add a filter, creating a cost allocation that only applies where `environment` is `production`.
+- **Dynamic by metric**: You can filter your metric query to focus on specific data. For example, add `environment:production` to your metric query so the allocation is based only on production environment usage data.
 
 ### Step 5 - (optional) Apply a partition