You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Reading the prompt actually sent to the LLM is the main way to debug generation — both defects in #47 were found that way. The only current route is the application log (LCMessageLogger.on_chat_model_start, src/generic_rag/utils/llm.py, attached in dial_client.py:315), which means digging through server logs to debug one chat request. That route is also going away: #51 restricts payload bodies in logs to a mode that is off by default.
A stage puts the prompt next to the response it produced, where the other debug output already lives — ChannelCompletion (src/generic_rag/app/chat_completion.py) already emits [DEBUG] channel configuration, [DEBUG] request configuration, the retrieval stages, and [DEBUG] token usage.
Proposal
Emit [DEBUG] generation prompt with the full prompt handed to the LLM: system message, user query, and the <context> block with its per-chunk attributes.
Gate it behind a debug environment variable, off by default — the prompt contains document content and the user's query. Note ChannelCompletion._enable_debug_output is a class constant hardcoded to True (chat_completion.py:31); this needs a real env-var switch, not a second hardcoded flag.
Notes
Redact image blocks — a base64 page image is tens of kilobytes and would flood the stage. Use a placeholder with mime type and payload size, e.g. data:image/png;base64,<5362 chars redacted>.
Render the prompt as actually sent (roles and content blocks in order), not a re-derived approximation.
Emit the assembled generation prompt as a DIAL stage, gated by a debug environment variable, off by default.
Split out from #47 (point 4).
Motivation
Reading the prompt actually sent to the LLM is the main way to debug generation — both defects in #47 were found that way. The only current route is the application log (
LCMessageLogger.on_chat_model_start,src/generic_rag/utils/llm.py, attached indial_client.py:315), which means digging through server logs to debug one chat request. That route is also going away: #51 restricts payload bodies in logs to a mode that is off by default.A stage puts the prompt next to the response it produced, where the other debug output already lives —
ChannelCompletion(src/generic_rag/app/chat_completion.py) already emits[DEBUG] channel configuration,[DEBUG] request configuration, the retrieval stages, and[DEBUG] token usage.Proposal
Emit
[DEBUG] generation promptwith the full prompt handed to the LLM: system message, user query, and the<context>block with its per-chunk attributes.Gate it behind a debug environment variable, off by default — the prompt contains document content and the user's query. Note
ChannelCompletion._enable_debug_outputis a class constant hardcoded toTrue(chat_completion.py:31); this needs a real env-var switch, not a second hardcoded flag.Notes
data:image/png;base64,<5362 chars redacted>.