diff --git a/content/en/llm_observability/sdk/_index.md b/content/en/llm_observability/sdk/_index.md
index 41a3729a7ddfe..5eef71e9bf126 100644
--- a/content/en/llm_observability/sdk/_index.md
+++ b/content/en/llm_observability/sdk/_index.md
@@ -1234,6 +1234,10 @@ The `LLMObs.submit_evaluation_for()` method accepts the following arguments:
`tags`
: optional - _dictionary_
A dictionary of string key-value pairs that users can add as tags regarding the evaluation. For more information about tags, see [Getting Started with Tags][2].
+
+`metadata`
+: optional - _dictionary_
+
A JSON serializable dictionary of key-value metadata pairs relevant to the evaluation metric.
{{% /collapse-content %}}
#### Example
@@ -1262,6 +1266,7 @@ def llm_call():
metric_type="score",
value=10,
tags={"evaluation_provider": "ragas"},
+ metadata={"flagged_segments": ["harmful part of output", "some other harmful part of output"]}
)
# joining an evaluation to a span via span ID and trace ID
@@ -1273,6 +1278,7 @@ def llm_call():
metric_type="score",
value=10,
tags={"evaluation_provider": "ragas"},
+ metadata={"flagged_segments": ["harmful part of output", "some other harmful part of output"]}
)
return completion
{{< /code-block >}}
@@ -1314,6 +1320,10 @@ The `evaluationOptions` object can contain the following:
`tags`
: optional - _dictionary_
A dictionary of string key-value pairs that users can add as tags regarding the evaluation. For more information about tags, see [Getting Started with Tags][1].
+
+`metadata`
+: optional - _dictionary_
+
A JSON serializable dictionary of key-value metadata pairs relevant to the evaluation metric.
{{% /collapse-content %}}
#### Example
@@ -1326,7 +1336,8 @@ function llmCall () {
label: "harmfulness",
metricType: "score",
value: 10,
- tags: { evaluationProvider: "ragas" }
+ tags: { evaluationProvider: "ragas" },
+ metadata: { flaggedSegments: ["harmful part of output", "some other harmful part of output"] }
})
return completion
}
diff --git a/content/en/llm_observability/setup/api.md b/content/en/llm_observability/setup/api.md
index edc3f5fafc290..3ef16ab207b7d 100644
--- a/content/en/llm_observability/setup/api.md
+++ b/content/en/llm_observability/setup/api.md
@@ -348,7 +348,10 @@ Evaluations must be joined to a unique span. You can identify the target span us
"timestamp_ms": 1609479200,
"metric_type": "score",
"label": "Accuracy",
- "score_value": 3
+ "score_value": 3,
+ "metadata": {
+ "flagged_segments": ["harmful part of output", "some other harmful part of output"]
+ }
}
]
}
@@ -380,6 +383,7 @@ Evaluations must be joined to a unique span. You can identify the target span us
| categorical_value [*required if the metric_type is "categorical"*] | string | A string representing the category that the evaluation belongs to. |
| score_value [*required if the metric_type is "score"*] | number | A score value of the evaluation. |
| tags | [[Tag](#tag)] | A list of tags to apply to this particular evaluation metric. |
+| metadata | Dict | Additional data relevant to the evaluation. |
#### JoinOn