[Azure OpenAI] Add API Management Gateway Logs Missing GatewayLlmLogs

[Mikaayenson]

Integration Name

Azure OpenAI [azure_openai]

Dataset Name

logs

Integration Version

1.9.0

Agent Version

8.17.1

Agent Output Type

elasticsearch

Elasticsearch Version

Serverless

OS Version and Architecture

N/A

Software/API Version

No response

Error Message

N/A

Event Original

None: The main issue is that the GatewayLlmLogs logs are dropped before getting to Kibana. Unlike the ChatCompletions API, when using the Responses API the input can by a List Input Item.

E.g.

Instead of input being a simple string, you can provide a list.

This will produce a GatewayLogs doc.

data = {
    "model":"gpt-5",
    "input": "tell me the top 3 best PS5 controllers locally"
}

This will produce a GatewayLlmLogs doc.

data = {
    "model": "gpt-5",
    "input": [
        { "role": "user",
          "content": [ { "type": "input_text", "text": "tell me the top 3 best PS5 controllers locally" } ] }
    ]
}

What did you do?

I ended up bypassing Elastic's integration and connected directly to the eventhub to see the raw docs generated. Here is a sample:

{"records": [{ "Region": "eastus", "serviceName": "trade-api-management", "Sku": "StandardV2", "Level": 4, "resourceId": "<redacted>", "time": "2025-10-31T06:29:06.4114107Z", "operationName": "Microsoft.ApiManagement/ApiManagementGatewayLlmLogs", "correlationId": "7d48f89b-19bf-40e6-b88c-243dc9d8df82", "category": "GatewayLlmLogs", "properties": {"timestamp":"2025-10-31T06:29:06.4114107Z","sequenceNumber":1,"promptTokens":0,"completionTokens":0,"totalTokens":0,"modelName":null,"isStreamCompletion":false,"requestId":null,"deploymentName":null,"apiVersion":null,"requestMessages":"{\"instructions\":\"You are a helpful assistant. Respond with a friendly greeting and confirm you can communicate.\",\"input\":[{\"type\":\"message\",\"content\":\"tell me the top 2 best PS5 games\",\"role\":\"user\"}]}","responseMessages":null}},{ "Region": "eastus", "serviceName": "trade-api-management", "Sku": "StandardV2", "Level": 4, "resourceId": "<redacted>", "time": "2025-10-31T06:29:06.4114107Z", "operationName": "Microsoft.ApiManagement/ApiManagementGatewayLlmLogs", "correlationId": "7d48f89b-19bf-40e6-b88c-243dc9d8df82", "category": "GatewayLlmLogs", "properties": {"timestamp":"2025-10-31T06:29:06.4114107Z","sequenceNumber":2,"promptTokens":0,"completionTokens":0,"totalTokens":0,"modelName":null,"isStreamCompletion":false,"requestId":null,"deploymentName":null,"apiVersion":null,"requestMessages":null,"responseMessages":"[{\"type\":\"reasoning\",\"id\":\"rs_0749fbed91561cd20069045732d5b081968afcd24f4a549c50\"},{\"type\":\"message\",\"id\":\"msg_0749fbed91561cd20069045763a96c819695aa27405abdcdf3\",\"status\":\"completed\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hi! I\\u2019m here and can communicate clearly.\\n\\nTop 2 PS5 games:\\n- Baldur\\u2019s Gate 3 \\u2014 a deep, choice-driven RPG with incredible writing, replayability, and co-op.\\n- Elden Ring \\u2014 a vast, challenging open-world adventure with unmatched freedom and discovery.\\n\\nIf you want only PlayStation exclusives, go with God of War Ragnar\\u00F6k and Marvel\\u2019s Spider-Man 2.\"}],\"role\":\"assistant\"}]"}},{ "Region": "eastus", "serviceName": "trade-api-management", "Sku": "StandardV2", "Level": 4, "resourceId": "<redacted>", "time": "2025-10-31T06:29:06.4114107Z", "location": "East US", "operationName": "Microsoft.ApiManagement/ApiManagementGatewayLlmLogs", "correlationId": "7d48f89b-19bf-40e6-b88c-243dc9d8df82", "category": "GatewayLlmLogs", "properties": {"timestamp":"2025-10-31T06:29:06.4114107Z","sequenceNumber":0,"promptTokens":37,"completionTokens":2266,"totalTokens":2303,"modelName":"gpt-5","isStreamCompletion":false,"requestId":"resp_0749fbed91561cd2006904573277048196a012e85462ea55d9","deploymentName":"gpt-5","apiVersion":"2025-04-01-preview","requestMessages":null,"responseMessages":null}}]}

What did you see?

I see docs when the category is 'RequestResponse' or 'GatewayLogs', just not GatewayLlmLogs

What did you expect to see?

Logs for GatewayLlmLogs populated in the stack.

Anything else?

Originally added in #10394 we can see fields dropped if they are not one of the three https://github.com/muthu-mps/integrations/blob/main/packages/azure_openai/data_stream/logs/elasticsearch/ingest_pipeline/default.yml#L24

[muthu-mps]

For more context around Logging LLM messages under (GatewayLlm log) category. This category was added recently to the API Management Services. While we explored the logs, we have noticed that it doesn't include the model_name to the log events. Without model name, these events with request and response text doesn't add any value. Only the token usage event has the model name. We have started a conversation around this with Azure team to include model name to all the events.

Ideally, what we can do is we can double check whether model name field populate values now, If not then we can get it closed with Azure team before including this events. WDYT?

[Mikaayenson]

@muthu-mps I think this is expected. GatewayLlmLogs emit multiple records per call and on the the summary/metrics record (usually sequenceNumber:0) carries modelName. We might be able to populate that information from the request as a workaround?

[muthu-mps]

@muthu-mps I think this is expected. GatewayLlmLogs emit multiple records per call and on the the summary/metrics record (usually sequenceNumber:0) carries modelName. We might be able to populate that information from the request as a workaround?

Based on the current observations of the log sample, the logs in the GatewayLlmLogs category cannot be directly processed by including the category name. Additional efforts are required. Adding @daniela-elastic for prioritizing the task.