Skip to content

Emit the assembled generation prompt as a DIAL stage behind a debug env var #50

Description

@navalnica

Emit the assembled generation prompt as a DIAL stage, gated by a debug environment variable, off by default.

Split out from #47 (point 4).

Motivation

Reading the prompt actually sent to the LLM is the main way to debug generation — both defects in #47 were found that way. The only current route is the application log (LCMessageLogger.on_chat_model_start, src/generic_rag/utils/llm.py, attached in dial_client.py:315), which means digging through server logs to debug one chat request. That route is also going away: #51 restricts payload bodies in logs to a mode that is off by default.

A stage puts the prompt next to the response it produced, where the other debug output already lives — ChannelCompletion (src/generic_rag/app/chat_completion.py) already emits [DEBUG] channel configuration, [DEBUG] request configuration, the retrieval stages, and [DEBUG] token usage.

Proposal

Emit [DEBUG] generation prompt with the full prompt handed to the LLM: system message, user query, and the <context> block with its per-chunk attributes.

Gate it behind a debug environment variable, off by default — the prompt contains document content and the user's query. Note ChannelCompletion._enable_debug_output is a class constant hardcoded to True (chat_completion.py:31); this needs a real env-var switch, not a second hardcoded flag.

Notes

  • Redact image blocks — a base64 page image is tens of kilobytes and would flood the stage. Use a placeholder with mime type and payload size, e.g. data:image/png;base64,<5362 chars redacted>.
  • Render the prompt as actually sent (roles and content blocks in order), not a re-derived approximation.
  • Consider governing this and Follow the logging policy: do not log sensitive payloads #51's payload logging with the same switch, so a deployment has one place to turn payload exposure on and off.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions