Issue

Lightning needs to be updated to display streaming text and status updates from the Apollo AI server. This will allow users to see progress instead of waiting for the complete response and enable long-running operations without timeout.

The AI assistant on the Apollo server now supports streaming. See the Apollo PR: Add answer streaming #288.

The job code assistant is a higher priority than the faster workflow assistant.

Required Changes:

• Update ApolloClient in Lightning:
  • Check if changes needed to support websocket connections for streaming responses from Apollo. We might reference the CLI approach in kit/packages/cli/src/apollo/handler.ts which we used to test streaming capability.
• Display the new event logs from Apollo to the user:
  • Identify event type: currently only “CHUNK" and "STATUS" are relevant. “CODE” for code suggestions can be ignored as the front-end has not been built yet.
  • For CHUNK: Display streamed chunks in the chat interface in real-time
    • Identify code blocks: During streaming, we need to identify code blocks marked by backticks. These could theoretically be split across two chunks. We can follow how Apollo’s job_chat accumulates chunks to check for JSON blocks. Alternatively, we can add an event type in Apollo.
  • For STATUS: Show status updates (like "Researching...", "Thinking...") in a visually distinct way
• Changes required in lightning/lib/lightning_web/live/ai_assistant/component.ex ? Where else?

UI/UX Considerations:

• Status updates should be visually distinct from content chunks. These can appear as a) a thread, or b) a line of text that is updated and disappears after the answer is ready (where “Processing…” currently appears)
• Status update language should be easy to understand, sound worthwhile and they should not leave confusing gaps in the process. These should be adjusted in Apollo.
• Text should stream smoothly even though we're receiving chunks of varying sizes.
• Code inside the text answers could be streamed inside the existing code block box. Like the text bubble, the code box should expand to fit the code. Users should not see the backticks.