diff --git a/OUTERR-STREAMING-PLAN.md b/OUTERR-STREAMING-PLAN.md new file mode 100644 index 0000000000..666d0d369d --- /dev/null +++ b/OUTERR-STREAMING-PLAN.md @@ -0,0 +1,45 @@ +# Plan: Toggleable Live Stdout/Stderr Streaming in Codex CLI + +## Current Behavior and Constraints + +- `codex-rs/core/src/exec.rs` already emits `EventMsg::ExecCommandOutputDelta` via `read_capped`, streaming stdout/stderr chunks while a command runs. The deltas carry the `call_id`, stream discriminator, and raw bytes, so the backend is ready. +- The TUI ignores those deltas today (`tui/src/chatwidget.rs` drops them in `on_exec_command_output_delta`), and `ExecCell` only renders final `CommandOutput` once `ExecCommandEnd` arrives. While a command is running the UI shows only a spinner and the command header. +- The bottom status indicator (`tui/src/status_indicator_widget.rs`) has infrastructure for queued hints but no hook for live command output lines or toggle affordances. +- Output folding is handled inside `tui/src/exec_cell/render.rs` by `output_lines`, which trims to `TOOL_CALL_MAX_LINES`. We should reuse its conventions to keep styling consistent. + +## Goal for the “simple” version +Expose a lightweight toggle that lets the user reveal or hide the most recent stdout/stderr lines for the currently running exec call directly inside the existing active command cell. When hidden, keep the status line showing the latest line so users still see progress. The implementation should cap memory, obey the Stylize helpers, and avoid protocol changes. + +## Implementation Steps + +1. **Model live output inside `ExecCell`** + - Introduce a `LiveExecStream` helper in `tui/src/exec_cell/model.rs` that tracks buffered lines per stream (stdout/stderr), pending partial line segments, and a capped `VecDeque` (e.g., keep the last 200 wrapped lines combined). Provide `push_chunk(stream, &[u8])` that decodes via `String::from_utf8_lossy`, splits on `\n`, updates the per-stream buffers, and returns the newest rendered line for status updates. + - Extend `ExecCall` with `live: LiveExecStream` (initialized when the call is created) and a `show_live_output: bool` flag scoped to the cell. Add convenience methods (`append_live_chunk`, `latest_live_line`, `toggle_live_output`) that `ChatWidget` can call. + - Ensure `ExecCell::complete_call` clears the live buffer and resets the toggle so completed history entries never keep the transient data. + +2. **Handle streaming events in `ChatWidget`** + - Update `ChatWidget::on_exec_command_output_delta` in `tui/src/chatwidget.rs` to locate the active `ExecCell`, forward the chunk to `ExecCell::append_live_chunk`, request a redraw, and store the last emitted line (if any) so we can surface it in the status indicator. + - Because events may land while the exec cell is queued, guard against missing cells by falling back to the `running_commands` map and a short-lived cache keyed by `call_id`. + - When we capture a new visible line, call a new `BottomPane::set_status_live_line` helper that updates the status indicator (see step 3). + +3. **Render live output and status hints** + - Extend `ExecCell::command_display_lines` in `tui/src/exec_cell/render.rs` to append the capped live output block whenever the call is active and `show_live_output` is true. Reuse `word_wrap_line`, `prefix_lines`, and `ansi_escape_line` so styling matches final output. Prefix stdout lines with a dim angle-pipe like the final output uses; color stderr lines red for quick scanning. + - When the toggle is off but buffered output exists, render a single dim hint line (“live output hidden — press ctrl+r”) so users discover the shortcut without expanding accidentally. + - In `StatusIndicatorWidget`, add optional storage for the latest streamed line and draw it on a second line under the header (trimmed to the pane width). Add a method such as `set_live_output_preview(String)` that the chat widget calls whenever a new line arrives. Clear it when the command completes. + +4. **Wire up the toggle gesture** + - Add a `KeyBinding` constant (e.g., `CTRL_R`) in `tui/src/status_indicator_widget.rs`/`tui/src/chatwidget.rs`. In `ChatWidget::handle_key_event`, detect `ctrl+r` while a command is running and flip the active exec cell’s `show_live_output` flag. Request a redraw and update the status indicator hint text to reflect the new state (“ctrl+r hide output” vs “ctrl+r show output”). + - Surface the shortcut in the status indicator by appending the key hint line (using the existing `key_hint` helpers) so the UI communicates the toggle while a task is running. + - Reset the toggle state, status preview line, and key hint when handling `ExecCommandEnd`. + +5. **Testing & polish** + - Add unit coverage for `LiveExecStream::push_chunk` to ensure partial lines and stderr/stdout separation work, and that line caps drop the oldest content. + - Extend `tui/src/chatwidget/tests.rs` with a snapshot covering: (a) live output hidden (shows hint), and (b) live output visible (shows wrapped lines and hints). Update `cargo insta` snapshots if needed. + - Manually confirm that streamed output appears for local exec commands and for MCP-driven commands (the protocol already forwards deltas, but sanity-check via Playwright or a fake tool). + - Implementation runbook: after coding, run `just fmt`, `just fix -p codex-tui`, and `cargo test -p codex-tui` per repo conventions. + +## Open Questions / Follow-ups +- Decide the exact cap (lines vs bytes). Lines are easier for the TUI; start with ~200 lines and revisit if memory spikes. +- Multi-command turns: today an exec cell can queue multiple shell calls (e.g., exploring lists). Confirm whether we should allow toggling per call or only the most recent active one. The simple path is to scope the toggle to the active call only. +- For non-UTF8 output we currently lossily decode; if that becomes an issue we may need to surface a “[binary chunk]” placeholder later. +- Future enhancements could reuse the same buffer to back a scrollable pager overlay, or persist the streamed lines into history when the command ends. diff --git a/STDOUT-STREAMING.md b/STDOUT-STREAMING.md new file mode 100644 index 0000000000..a853b139e1 --- /dev/null +++ b/STDOUT-STREAMING.md @@ -0,0 +1,323 @@ +# Streaming stdout and stderr from long-running processes + +## https://github.com/openai/codex/issues/4751 + +What feature would you like to see? +I would like the ability to see the real-time stdout and stderr from long-running shell commands that Codex executes, especially those run via MCP tools. + +Currently, when Codex runs a command like npm run test:e2e or a large npm install, the TUI only displays a generic spinner (e.g., ⠏ Running npm run test:e2e). This provides no visibility into the progress, output, or potential errors of the underlying process. + +The ideal behavior would be for the output of that command to be streamed directly into the TUI, below the "Running..." indicator. This would allow users to: + +Monitor the progress of the task (e.g., see test suites running, files being compiled). +Debug issues in real-time if the command produces errors. +Know that the process has not hung or stalled. +This would significantly improve the user experience and build confidence that Codex is actively working on the task. + +Additional information +I encountered this issue while using the Playwright MCP to run end-to-end tests on a project. Codex correctly identified the task, created a plan, installed dependencies, and then initiated the test command: npm run test:e2e. + +At this point, the UI showed a spinner and the message ⠏ Running npm run test:e2e for over 20 minutes with no further updates or information. I had no way of knowing if the tests were running, stalled, or had failed. The Playwright test runner typically has a rich, detailed output showing which tests are running, passing, or failing, all of which was completely hidden. + +While my specific case was with Playwright, this feature is essential for any potentially long-running process, such as: + +Running database migrations (prisma migrate dev). +Complex build scripts (npm run build). +Large dependency installations (npm install). +Providing this visibility is critical for a tool designed for developers. + +TUI log showing the stall point: + +``` +› use the playwright mcp to extensively test this project. make a plan first. + +• Updated Plan + └ □ Review available Playwright tooling and existing test structure to understand required setup. + □ Install dependencies and ensure test environment prerequisites are satisfied. + □ Execute Playwright end-to-end suites via MCP integration with appropriate configurations. + □ Summarize results and note any observed failures or follow-ups. + +• Explored + └ List e2e + List ls + Read playwright.config.ts, playwright-global-setup.ts + Search MCP + +• Updated Plan + └ ✔ Review available Playwright tooling and existing test structure to understand required setup. + □ Install dependencies and ensure test environment prerequisites are satisfied. + □ Execute Playwright end-to-end suites via MCP integration with appropriate configurations. + □ Summarize results and note any observed failures or follow-ups. + +• Explored + └ Read env.setup.js + List ls -a + Read .env.test + List supabase + Read package.json + +• Ran node -v + └ v24.8.0 + +• Ran npm install + └ + added 14 packages, and audited 1113 packages in 2s + … +3 lines + + found 0 vulnerabilities + +• Explored + └ Read .env + +• I’m considering the implications of running tests that connect to a remote Supabase instance using provided + credentials, which might modify real data. I want to ensure safe setup by verifying connectivity without + destructive actions and confirming necessary dependencies like Playwright are installed before proceeding. + +• Ran npx playwright install + └ (no output) + +• Updated Plan + └ ✔ Review available Playwright tooling and existing test structure to understand required setup. + ✔ Install dependencies and ensure test environment prerequisites are satisfied. + □ Execute Playwright end-to-end suites via MCP integration with appropriate configurations. + □ Summarize results and note any observed failures or follow-ups. + +• Explored + └ Read video-features-mcp-headless.spec.ts + +• I’m making sure the app server and Supabase environment are properly set up for running npm run test:e2e. The + tests rely on remote Supabase for login and API routes, so environment variables from .env must be loaded. I'll + run all tests across browsers, noting they might be time-consuming but should pass since some tests auto-pass + while others expect manual steps. + +⠏ Running npm run test:e2e + +• Preparing to run Playwright tests (23m 49s • esc to interrupt) +``` + +### Follow-up comment [to 4751](https://github.com/openai/codex/issues/4751#issuecomment-3368703897) + +Here is a potential high-level architectural plan to implement real-time output streaming for shell commands in the Codex TUI. + +Architectural Plan: Real-Time Command Output Streaming +1. Guiding Principles +Decoupling: The solution should be layered. The command execution logic should not know about the TUI's rendering details. Communication should happen via a well-defined event protocol. +Generality: The streaming mechanism should work for both local shell commands (local_shell) and commands executed via MCP tools. The TUI should not need to differentiate between them. +Efficiency: We must stream output in chunks without waiting for the command to complete, but also without overwhelming the TUI with too many render updates. +2. High-Level Overview +The core idea is to change the command execution flow from a synchronous "request-response" model (where the full output is returned at the end) to an asynchronous, event-driven streaming model. + +This involves three main areas of work: + +Core Execution Layer: Modify the command execution logic to capture and stream stdout and stderr chunks in real-time. +Protocol Layer: Introduce a new event type to carry these real-time output chunks from the execution layer back to the UI. +TUI Presentation Layer: Update the TUI's history and active command cells to receive and render these streaming updates. +3. Detailed Component Breakdown +Component 1: Core Execution Layer (codex-core) +This is where the command process is spawned and its output is read. + +Introduce a New Protocol Event: + +In codex-rs/core/src/protocol/protocol.rs, we will define a new event, ExecCommandOutputDeltaEvent. +This event will contain: +call_id: The unique identifier for the specific tool call, allowing the TUI to associate the output with the correct running command. +stream: An enum indicating the source (stdout or stderr). +data: The chunk of output as a String or Vec. +Modify the Execution Pipeline: + +The current function process_exec_tool_call in codex-rs/core/src/exec.rs waits for the command to finish and returns a complete ExecToolCallOutput. This needs to be adapted. +We will introduce an optional event_sender (a tokio::sync::mpsc::Sender) into the execution context, likely passed into process_exec_tool_call via a new field in the StdoutStream struct. +The read_capped function, which already reads from the child process's streams in a separate task, is the ideal place to implement the streaming. Instead of just appending to a buffer, it will now also send ExecCommandOutputDeltaEvents through the provided channel for each chunk it reads. +Update Final Output: + +The final ExecCommandEndEvent will still be sent when the command completes. Its payload (stdout, stderr, aggregated_output) will contain the full captured output, ensuring that even if the TUI misses some delta events (e.g., due to high volume), the final state is consistent. The live streaming is for UX, while the final event is for the permanent record. +Component 2: TUI Presentation Layer (codex-tui) +This layer will receive the new events and render the live output. + +Handle the New Event: + +In codex-rs/tui/src/chatwidget.rs, the main event loop (ChatWidget::handle_codex_event) will get a new case for EventMsg::ExecCommandOutputDelta. +Update the Active Command Cell Model (ExecCell): + +The ExecCell struct in codex-rs/tui/src/exec_cell/model.rs, which represents a command execution in the UI, currently only holds the final CommandOutput. +We will add a new field to ExecCell to buffer the live, streamed output lines, for example: live_output: Vec>. This buffer will be populated as ExecCommandOutputDeltaEvents arrive. +Enhance Rendering Logic: + +The rendering logic in codex-rs/tui/src/exec_cell/render.rs for an active ExecCell (one that is still running) will be updated. +Currently, it just shows a spinner and the command. The new logic will render the contents of the live_output buffer directly below the command line, effectively creating a mini-terminal view for the running command. +We will implement a capped-size circular buffer for live_output to prevent memory issues from extremely verbose commands, showing only the most recent N lines. +Event Flow in the TUI: + +When ExecCommandBeginEvent arrives, an ExecCell is created and stored as the active_cell in ChatWidget. +As ExecCommandOutputDeltaEvent events arrive, ChatWidget finds the active_cell, downcasts it to ExecCell, and appends the new output lines to its live_output buffer. It then requests a TUI redraw. +When ExecCommandEndEvent arrives, the active_cell is finalized with the full output and exit code, and it is moved from the active_cell slot into the permanent history log (transcript_cells). +Component 3: MCP Tool Integration (Generalization) +To support tools like Playwright running over MCP, the streaming mechanism must be extended. + +MCP Server (codex-mcp-server): + +The codex_tool_runner which spawns and manages the CodexConversation for a tools/call request will listen for the new ExecCommandOutputDeltaEvent. +Upon receiving this event, it will forward the data to the connected MCP client using an appropriate MCP notification, likely a tool/stream notification type if the spec supports it, or a custom codex/exec_output_delta notification. +MCP Client (codex-core): + +The McpConnectionManager will be updated to listen for these new streaming notifications from the server. +When it receives a streaming notification, it will transform it back into a local ExecCommandOutputDeltaEvent and send it to the Session's event queue (tx_event). +Result: From the TUI's perspective, there is no difference. It receives an ExecCommandOutputDeltaEvent regardless of whether the command was executed by the local shell or a remote MCP server, achieving the desired generality. + +4. Implementation Stages +Protocol First: Define ExecCommandOutputDeltaEvent in codex-protocol and codex-core. +Local Shell Implementation: Implement the streaming logic for local commands first. This involves modifying codex-core/src/exec.rs and the TUI's ExecCell to handle and display the new event. This provides the core functionality quickly. +TUI Presentation: Update the ChatWidget and ExecCell to render the streamed output, including handling line wrapping and scrolling within the active cell. +MCP Extension: Extend the functionality to the MCP server and client layers to enable streaming for all tool types. +This plan establishes a robust pipeline for real-time data, significantly enhancing the user experience by providing the requested visibility into long-running commands. + +## https://github.com/openai/codex/issues/3675 + +What feature would you like to see? +Show stdout/stderr of commands run by the shell tool. Its a lot easier to get an idea of what an LLM is doing behind the scenes that way. + +Would be great if it's implemented kind of like how claude code does it (truncated by default but you can press Ctrl + R to expand). I wouldn't even mind not having the expand features just a greyed out truncated output is good enough. + +## https://github.com/openai/codex/issues/4179 + +What feature would you like to see? +Summary + +Add a non-interactive, headless “supervise & repair” mode to Codex CLI that continuously stabilizes a running app by orchestrating Playwright MCP to simulate real users, watching live logs, auto-diagnosing failures, applying fixes, and resuming the same user journey—repeating until a configurable zero-error stability goal is met (e.g., no critical errors for N minutes). + +Problem + +Long-running processes (dev servers, Docker) and evolving features often break user flows. Codex currently tends to act after a command finishes, which doesn’t work for never-ending tasks and delays remediation. We need an automatic, under-the-hood loop that observes the app while it runs, drives a real browser like a user, fixes issues the moment they appear, and continues without human intervention. + +Desired Behavior + +Start or attach to the app process and stream stdout/stderr incrementally (no wait for process exit). + +Connect to Playwright MCP and execute live user flows (navigate, type, click, submit, validate). + +On any runtime failure (console errors, unhandled exceptions, HTTP 5xx/4xx thresholds, Playwright selector/timeouts, health-check failures), Codex: + +Captures artifacts (logs, screenshot, trace, DOM snapshot, network/HAR). + +Drafts a root-cause and a minimal, safe code change. + +Applies the fix automatically (policy-gated, no prompts). + +Hot-reloads or restarts the affected service if needed. + +Resumes the exact flow at the failing step. + +Repeats until stability goals are met or bounded retry/backoff limits are reached. + +Scope + +Automation only: runs headless; no UI/TUI. + +Playwright MCP as the browser driver; optional direct HTTP calls (via MCP tool) when UI interaction isn’t necessary. + +Supports multiple supervised processes (e.g., web, API, worker). + +Design Requirements + +Live log ingestion: unbuffered, partial-line reads with lightweight summarization and debounced trigger evaluation. + +Trigger model: regex patterns, severity/keyword thresholds, JSON-structured log queries, and MCP/Playwright events (timeouts, “not found,” navigation errors). + +Repair policy: apply only minimal, reversible diffs; keep a rollback plan; bound changes per cycle (e.g., max lines changed/files touched). + +Resume semantics: restart from the same Playwright step with state restoration (storage state, cookies, auth), or rerun the smallest stable sub-flow. + +Retry/backoff: progressive delays, circuit-breaker after repeated failure modes, clear stop conditions. + +Stability goals: configurable targets (e.g., “zero critical errors for 5 minutes,” “no console.error above allowlist,” “network-healthy: 0% 5xx and , context_window_percent: Option, + status_live_line: Option, + live_output_visible: bool, + live_output_toggle_enabled: bool, } pub(crate) struct BottomPaneParams { @@ -102,6 +105,9 @@ impl BottomPane { queued_user_messages: Vec::new(), esc_backtrack_hint: false, context_window_percent: None, + status_live_line: None, + live_output_visible: false, + live_output_toggle_enabled: false, } } @@ -335,11 +341,17 @@ impl BottomPane { } if let Some(status) = self.status.as_mut() { status.set_queued_messages(self.queued_user_messages.clone()); + status.set_live_output_preview(self.status_live_line.clone()); + status.set_live_output_visible(self.live_output_visible); + status.set_live_output_toggle_enabled(self.live_output_toggle_enabled); } self.request_redraw(); } else { // Hide the status indicator when a task completes, but keep other modal views. self.hide_status_indicator(); + self.status_live_line = None; + self.live_output_visible = false; + self.live_output_toggle_enabled = false; } } @@ -375,6 +387,39 @@ impl BottomPane { self.request_redraw(); } + pub(crate) fn set_status_live_line(&mut self, line: Option) { + if self.status_live_line == line { + return; + } + self.status_live_line = line.clone(); + if let Some(status) = self.status.as_mut() { + status.set_live_output_preview(line); + } + self.request_redraw(); + } + + pub(crate) fn set_live_output_visible(&mut self, visible: bool) { + if self.live_output_visible == visible { + return; + } + self.live_output_visible = visible; + if let Some(status) = self.status.as_mut() { + status.set_live_output_visible(visible); + } + self.request_redraw(); + } + + pub(crate) fn set_live_output_toggle_enabled(&mut self, enabled: bool) { + if self.live_output_toggle_enabled == enabled { + return; + } + self.live_output_toggle_enabled = enabled; + if let Some(status) = self.status.as_mut() { + status.set_live_output_toggle_enabled(enabled); + } + self.request_redraw(); + } + /// Update custom prompts available for the slash popup. pub(crate) fn set_custom_prompts(&mut self, prompts: Vec) { self.composer.set_custom_prompts(prompts); diff --git a/codex-rs/tui/src/chatwidget.rs b/codex-rs/tui/src/chatwidget.rs index 973b0e3ba2..690c87e7de 100644 --- a/codex-rs/tui/src/chatwidget.rs +++ b/codex-rs/tui/src/chatwidget.rs @@ -21,6 +21,7 @@ use codex_core::protocol::EventMsg; use codex_core::protocol::ExecApprovalRequestEvent; use codex_core::protocol::ExecCommandBeginEvent; use codex_core::protocol::ExecCommandEndEvent; +use codex_core::protocol::ExecCommandOutputDeltaEvent; use codex_core::protocol::ExitedReviewModeEvent; use codex_core::protocol::InputItem; use codex_core::protocol::InputMessageKind; @@ -386,6 +387,9 @@ impl ChatWidget { fn on_task_started(&mut self) { self.bottom_pane.clear_ctrl_c_quit_hint(); self.bottom_pane.set_task_running(true); + self.bottom_pane.set_status_live_line(None); + self.bottom_pane.set_live_output_visible(false); + self.bottom_pane.set_live_output_toggle_enabled(false); self.full_reasoning_buffer.clear(); self.reasoning_buffer.clear(); self.request_redraw(); @@ -396,6 +400,9 @@ impl ChatWidget { self.flush_answer_stream_with_separator(); // Mark task stopped and request redraw now that all content is in history. self.bottom_pane.set_task_running(false); + self.bottom_pane.set_status_live_line(None); + self.bottom_pane.set_live_output_visible(false); + self.bottom_pane.set_live_output_toggle_enabled(false); self.running_commands.clear(); self.request_redraw(); @@ -537,11 +544,39 @@ impl ChatWidget { self.defer_or_handle(|q| q.push_exec_begin(ev), |s| s.handle_exec_begin_now(ev2)); } - fn on_exec_command_output_delta( - &mut self, - _ev: codex_core::protocol::ExecCommandOutputDeltaEvent, - ) { - // TODO: Handle streaming exec output if/when implemented + fn on_exec_command_output_delta(&mut self, ev: ExecCommandOutputDeltaEvent) { + let ExecCommandOutputDeltaEvent { + call_id, + stream, + chunk, + } = ev; + + if let Some(cell) = self + .active_cell + .as_mut() + .and_then(|c| c.as_any_mut().downcast_mut::()) + { + let preview = cell.append_live_chunk(&call_id, stream, &chunk); + let has_live = !cell.live_lines_for_display().is_empty(); + if let Some(text) = preview.or_else(|| cell.latest_live_preview()) { + self.bottom_pane.set_status_live_line(Some(text)); + } + if has_live { + self.bottom_pane.set_live_output_toggle_enabled(true); + } + self.bottom_pane + .set_live_output_visible(cell.show_live_output()); + self.request_redraw(); + return; + } + + if !chunk.is_empty() { + let fallback = String::from_utf8_lossy(&chunk).to_string(); + if !fallback.is_empty() { + self.bottom_pane + .set_status_live_line(Some(fallback.trim_end_matches('\n').to_string())); + } + } } fn on_patch_apply_begin(&mut self, event: PatchApplyBeginEvent) { @@ -739,6 +774,9 @@ impl ChatWidget { self.flush_active_cell(); } } + self.bottom_pane.set_status_live_line(None); + self.bottom_pane.set_live_output_visible(false); + self.bottom_pane.set_live_output_toggle_enabled(false); } pub(crate) fn handle_patch_apply_end_now( @@ -818,6 +856,17 @@ impl ChatWidget { ))); } + if let Some(cell) = self + .active_cell + .as_mut() + .and_then(|c| c.as_any_mut().downcast_mut::()) + { + cell.set_live_output_visible(false); + } + self.bottom_pane.set_status_live_line(None); + self.bottom_pane.set_live_output_visible(false); + self.bottom_pane.set_live_output_toggle_enabled(true); + self.request_redraw(); } @@ -1034,6 +1083,16 @@ impl ChatWidget { } return; } + KeyEvent { + code: KeyCode::Char('r'), + modifiers: KeyModifiers::CONTROL, + kind: KeyEventKind::Press, + .. + } => { + if self.toggle_live_output_view() { + return; + } + } other if other.kind == KeyEventKind::Press => { self.bottom_pane.clear_ctrl_c_quit_hint(); } @@ -1232,6 +1291,27 @@ impl ChatWidget { } } + fn toggle_live_output_view(&mut self) -> bool { + let Some(cell) = self + .active_cell + .as_mut() + .and_then(|c| c.as_any_mut().downcast_mut::()) + else { + return false; + }; + if !cell.is_active() { + return false; + } + let new_state = cell.toggle_live_output(); + self.bottom_pane.set_live_output_visible(new_state); + self.bottom_pane.set_live_output_toggle_enabled(true); + if let Some(preview) = cell.latest_live_preview() { + self.bottom_pane.set_status_live_line(Some(preview)); + } + self.request_redraw(); + true + } + fn add_to_history(&mut self, cell: impl HistoryCell + 'static) { self.add_boxed_history(Box::new(cell)); } diff --git a/codex-rs/tui/src/exec_cell/mod.rs b/codex-rs/tui/src/exec_cell/mod.rs index 906091113e..31bc588271 100644 --- a/codex-rs/tui/src/exec_cell/mod.rs +++ b/codex-rs/tui/src/exec_cell/mod.rs @@ -5,6 +5,8 @@ pub(crate) use model::CommandOutput; #[cfg(test)] pub(crate) use model::ExecCall; pub(crate) use model::ExecCell; +#[cfg(test)] +pub(crate) use model::LiveExecStream; pub(crate) use render::OutputLinesParams; pub(crate) use render::TOOL_CALL_MAX_LINES; pub(crate) use render::new_active_exec_command; diff --git a/codex-rs/tui/src/exec_cell/model.rs b/codex-rs/tui/src/exec_cell/model.rs index 2893a005dc..3cfafebcd3 100644 --- a/codex-rs/tui/src/exec_cell/model.rs +++ b/codex-rs/tui/src/exec_cell/model.rs @@ -1,8 +1,124 @@ +use std::collections::VecDeque; use std::time::Duration; use std::time::Instant; +use codex_core::protocol::ExecOutputStream; use codex_protocol::parse_command::ParsedCommand; +const LIVE_OUTPUT_MAX_LINES: usize = 200; + +#[derive(Clone, Debug)] +pub(crate) struct LiveExecLine { + pub(crate) stream: ExecOutputStream, + pub(crate) text: String, +} + +#[derive(Clone, Debug)] +pub(crate) struct LiveExecStream { + lines: VecDeque, + pending_stdout: String, + pending_stderr: String, + pending_order: Vec, +} + +impl LiveExecStream { + pub(crate) fn new() -> Self { + Self { + lines: VecDeque::new(), + pending_stdout: String::new(), + pending_stderr: String::new(), + pending_order: Vec::new(), + } + } + + pub(crate) fn clear(&mut self) { + self.lines.clear(); + self.pending_stdout.clear(); + self.pending_stderr.clear(); + self.pending_order.clear(); + } + + pub(crate) fn has_content(&self) -> bool { + !self.lines.is_empty() || !self.pending_stdout.is_empty() || !self.pending_stderr.is_empty() + } + + pub(crate) fn push_chunk(&mut self, stream: ExecOutputStream, chunk: &[u8]) -> Option { + let text = String::from_utf8_lossy(chunk); + match stream { + ExecOutputStream::Stdout => self.pending_stdout.push_str(&text), + ExecOutputStream::Stderr => self.pending_stderr.push_str(&text), + } + self.touch_pending_order(&stream); + + let (completed_lines, pending_snapshot, pending_empty) = { + let pending = match stream { + ExecOutputStream::Stdout => &mut self.pending_stdout, + ExecOutputStream::Stderr => &mut self.pending_stderr, + }; + let mut completed: Vec = Vec::new(); + while let Some(idx) = pending.find('\n') { + let mut line: String = pending.drain(..=idx).collect(); + if line.ends_with('\n') { + line.pop(); + } + if line.ends_with('\r') { + line.pop(); + } + completed.push(line); + } + let pending_snapshot = pending.clone(); + let pending_empty = pending.is_empty(); + (completed, pending_snapshot, pending_empty) + }; + + if pending_empty { + self.pending_order.retain(|s| s != &stream); + } + + for line in &completed_lines { + self.push_line(stream.clone(), line.clone()); + } + + if let Some(last) = completed_lines.last() { + Some(last.clone()) + } else if !pending_snapshot.is_empty() { + Some(pending_snapshot) + } else { + None + } + } + + pub(crate) fn lines_for_display(&self) -> Vec { + let mut out: Vec = self.lines.iter().cloned().collect(); + for stream in &self.pending_order { + let pending = match stream { + ExecOutputStream::Stdout => &self.pending_stdout, + ExecOutputStream::Stderr => &self.pending_stderr, + }; + if pending.is_empty() { + continue; + } + out.push(LiveExecLine { + stream: stream.clone(), + text: pending.clone(), + }); + } + out + } + + fn push_line(&mut self, stream: ExecOutputStream, text: String) { + self.lines.push_back(LiveExecLine { stream, text }); + while self.lines.len() > LIVE_OUTPUT_MAX_LINES { + self.lines.pop_front(); + } + } + + fn touch_pending_order(&mut self, stream: &ExecOutputStream) { + self.pending_order.retain(|s| s != stream); + self.pending_order.push(stream.clone()); + } +} + #[derive(Clone, Debug)] pub(crate) struct CommandOutput { pub(crate) exit_code: i32, @@ -19,16 +135,21 @@ pub(crate) struct ExecCall { pub(crate) output: Option, pub(crate) start_time: Option, pub(crate) duration: Option, + pub(crate) live: LiveExecStream, } #[derive(Debug)] pub(crate) struct ExecCell { pub(crate) calls: Vec, + show_live_output: bool, } impl ExecCell { pub(crate) fn new(call: ExecCall) -> Self { - Self { calls: vec![call] } + Self { + calls: vec![call], + show_live_output: false, + } } pub(crate) fn with_added_call( @@ -44,10 +165,12 @@ impl ExecCell { output: None, start_time: Some(Instant::now()), duration: None, + live: LiveExecStream::new(), }; if self.is_exploring_cell() && Self::is_exploring_call(&call) { Some(Self { calls: [self.calls.clone(), vec![call]].concat(), + show_live_output: false, }) } else { None @@ -64,6 +187,8 @@ impl ExecCell { call.output = Some(output); call.duration = Some(duration); call.start_time = None; + call.live.clear(); + self.show_live_output = false; } } @@ -86,8 +211,10 @@ impl ExecCell { stderr: String::new(), formatted_output: String::new(), }); + call.live.clear(); } } + self.show_live_output = false; } pub(crate) fn is_exploring_cell(&self) -> bool { @@ -109,6 +236,57 @@ impl ExecCell { self.calls.iter() } + pub(crate) fn append_live_chunk( + &mut self, + call_id: &str, + stream: ExecOutputStream, + chunk: &[u8], + ) -> Option { + if let Some(call) = self.calls.iter_mut().rev().find(|c| c.call_id == call_id) { + return call.live.push_chunk(stream, chunk); + } + None + } + + pub(crate) fn toggle_live_output(&mut self) -> bool { + self.show_live_output = !self.show_live_output; + self.show_live_output + } + + pub(crate) fn set_live_output_visible(&mut self, visible: bool) { + self.show_live_output = visible; + } + + pub(crate) fn show_live_output(&self) -> bool { + self.show_live_output + } + + pub(crate) fn latest_live_preview(&self) -> Option { + self.calls + .iter() + .rev() + .find(|c| c.output.is_none()) + .and_then(|call| { + if !call.live.has_content() { + None + } else { + call.live + .lines_for_display() + .last() + .map(|line| line.text.clone()) + } + }) + } + + pub(crate) fn live_lines_for_display(&self) -> Vec { + self.calls + .iter() + .rev() + .find(|c| c.output.is_none()) + .map(|call| call.live.lines_for_display()) + .unwrap_or_default() + } + pub(super) fn is_exploring_call(call: &ExecCall) -> bool { !call.parsed.is_empty() && call.parsed.iter().all(|p| { @@ -121,3 +299,42 @@ impl ExecCell { }) } } + +#[cfg(test)] +mod tests { + use super::*; + use pretty_assertions::assert_eq; + + #[test] + fn live_exec_stream_collects_lines() { + let mut stream = LiveExecStream::new(); + let preview = stream.push_chunk(ExecOutputStream::Stdout, b"foo"); + assert_eq!(preview.as_deref(), Some("foo")); + let mut lines = stream.lines_for_display(); + assert_eq!(lines.len(), 1); + assert_eq!(lines[0].text, "foo"); + + let preview = stream.push_chunk(ExecOutputStream::Stdout, b"bar\nbaz\n"); + assert_eq!(preview.as_deref(), Some("baz")); + lines = stream.lines_for_display(); + assert_eq!(lines.len(), 2); + assert_eq!(lines[0].text, "foobar"); + assert_eq!(lines[1].text, "baz"); + } + + #[test] + fn live_exec_stream_caps_lines() { + let mut stream = LiveExecStream::new(); + for i in 0..(LIVE_OUTPUT_MAX_LINES + 5) { + let line = format!("line-{i}\n"); + let _ = stream.push_chunk(ExecOutputStream::Stdout, line.as_bytes()); + } + let lines = stream.lines_for_display(); + assert_eq!(lines.len(), LIVE_OUTPUT_MAX_LINES); + assert_eq!(lines.first().unwrap().text, "line-5"); + assert_eq!( + lines.last().unwrap().text, + format!("line-{}", LIVE_OUTPUT_MAX_LINES + 4) + ); + } +} diff --git a/codex-rs/tui/src/exec_cell/render.rs b/codex-rs/tui/src/exec_cell/render.rs index a3cc8cacae..f51d2b92d3 100644 --- a/codex-rs/tui/src/exec_cell/render.rs +++ b/codex-rs/tui/src/exec_cell/render.rs @@ -3,6 +3,7 @@ use std::time::Instant; use super::model::CommandOutput; use super::model::ExecCall; use super::model::ExecCell; +use super::model::LiveExecStream; use crate::exec_command::strip_bash_lc_and_escape; use crate::history_cell::HistoryCell; use crate::render::highlight::highlight_bash_to_lines; @@ -14,10 +15,13 @@ use crate::wrapping::word_wrap_line; use crate::wrapping::word_wrap_lines; use codex_ansi_escape::ansi_escape_line; use codex_common::elapsed::format_duration; +use codex_core::protocol::ExecOutputStream; use codex_protocol::parse_command::ParsedCommand; use itertools::Itertools; use ratatui::prelude::*; +use ratatui::style::Color; use ratatui::style::Modifier; +use ratatui::style::Style; use ratatui::style::Stylize; use ratatui::widgets::Paragraph; use ratatui::widgets::WidgetRef; @@ -45,6 +49,7 @@ pub(crate) fn new_active_exec_command( output: None, start_time: Some(Instant::now()), duration: None, + live: LiveExecStream::new(), }) } @@ -356,6 +361,10 @@ impl ExecCell { } } + let output_wrap_width = layout.output_block.wrap_width(width); + let output_opts = + RtOptions::new(output_wrap_width).word_splitter(WordSplitter::NoHyphenation); + let mut lines: Vec> = vec![header_line]; let continuation_lines = Self::limit_lines_from_start( @@ -370,6 +379,42 @@ impl ExecCell { )); } + if call.output.is_none() { + let live_lines = self.live_lines_for_display(); + if !live_lines.is_empty() { + if self.show_live_output() { + let mut rendered_live: Vec> = Vec::new(); + for live in live_lines { + let mut line = ansi_escape_line(&live.text); + let style = match live.stream { + ExecOutputStream::Stdout => Style::default().dim(), + ExecOutputStream::Stderr => Style::default().fg(Color::Red), + }; + for span in line.spans.iter_mut() { + span.style = span.style.patch(style); + } + push_owned_lines( + &word_wrap_line(&line, output_opts.clone()), + &mut rendered_live, + ); + } + if !rendered_live.is_empty() { + lines.extend(prefix_lines( + rendered_live, + Span::from(layout.output_block.initial_prefix).dim(), + Span::from(layout.output_block.subsequent_prefix), + )); + } + } else { + lines.extend(prefix_lines( + vec![Line::from("live output hidden — press ctrl+r".dim())], + Span::from(layout.output_block.initial_prefix).dim(), + Span::from(layout.output_block.subsequent_prefix), + )); + } + } + } + if let Some(output) = call.output.as_ref() { let raw_output_lines = output_lines( Some(output), @@ -391,9 +436,6 @@ impl ExecCell { Self::truncate_lines_middle(&raw_output_lines, layout.output_max_lines); let mut wrapped_output: Vec> = Vec::new(); - let output_wrap_width = layout.output_block.wrap_width(width); - let output_opts = - RtOptions::new(output_wrap_width).word_splitter(WordSplitter::NoHyphenation); for line in trimmed_output { push_owned_lines( &word_wrap_line(&line, output_opts.clone()), diff --git a/codex-rs/tui/src/history_cell.rs b/codex-rs/tui/src/history_cell.rs index 18a6c72d24..70a582183c 100644 --- a/codex-rs/tui/src/history_cell.rs +++ b/codex-rs/tui/src/history_cell.rs @@ -1278,6 +1278,7 @@ mod tests { use crate::exec_cell::CommandOutput; use crate::exec_cell::ExecCall; use crate::exec_cell::ExecCell; + use crate::exec_cell::LiveExecStream; use codex_core::config::Config; use codex_core::config::ConfigOverrides; use codex_core::config::ConfigToml; @@ -1599,6 +1600,7 @@ mod tests { output: None, start_time: Some(Instant::now()), duration: None, + live: LiveExecStream::new(), }); // Mark call complete so markers are ✓ cell.complete_call( @@ -1630,6 +1632,7 @@ mod tests { output: None, start_time: Some(Instant::now()), duration: None, + live: LiveExecStream::new(), }); // Call 1: Search only cell.complete_call( @@ -1712,6 +1715,7 @@ mod tests { output: None, start_time: Some(Instant::now()), duration: None, + live: LiveExecStream::new(), }); cell.complete_call( "c1", @@ -1740,6 +1744,7 @@ mod tests { output: None, start_time: Some(Instant::now()), duration: None, + live: LiveExecStream::new(), }); // Mark call complete so it renders as "Ran" cell.complete_call( @@ -1770,6 +1775,7 @@ mod tests { output: None, start_time: Some(Instant::now()), duration: None, + live: LiveExecStream::new(), }); cell.complete_call( &call_id, @@ -1798,6 +1804,7 @@ mod tests { output: None, start_time: Some(Instant::now()), duration: None, + live: LiveExecStream::new(), }); cell.complete_call( &call_id, @@ -1825,6 +1832,7 @@ mod tests { output: None, start_time: Some(Instant::now()), duration: None, + live: LiveExecStream::new(), }); cell.complete_call( &call_id, @@ -1853,6 +1861,7 @@ mod tests { output: None, start_time: Some(Instant::now()), duration: None, + live: LiveExecStream::new(), }); cell.complete_call( &call_id, @@ -1881,6 +1890,7 @@ mod tests { output: None, start_time: Some(Instant::now()), duration: None, + live: LiveExecStream::new(), }); let stderr: String = (1..=10) .map(|n| n.to_string()) @@ -1927,6 +1937,7 @@ mod tests { output: None, start_time: Some(Instant::now()), duration: None, + live: LiveExecStream::new(), }); let stderr = "error: first line on stderr\nerror: second line on stderr".to_string(); diff --git a/codex-rs/tui/src/status_indicator_widget.rs b/codex-rs/tui/src/status_indicator_widget.rs index ce4f6eabd8..d5249651fb 100644 --- a/codex-rs/tui/src/status_indicator_widget.rs +++ b/codex-rs/tui/src/status_indicator_widget.rs @@ -25,6 +25,12 @@ pub(crate) struct StatusIndicatorWidget { header: String, /// Queued user messages to display under the status line. queued_messages: Vec, + /// Most recent stdout/stderr line from the active exec call. + live_line: Option, + /// Whether the live output panel is expanded in the active exec cell. + live_output_visible: bool, + /// Whether ctrl+r toggle hint should be shown. + live_output_toggle_enabled: bool, elapsed_running: Duration, last_resume_at: Instant, @@ -55,6 +61,9 @@ impl StatusIndicatorWidget { Self { header: String::from("Working"), queued_messages: Vec::new(), + live_line: None, + live_output_visible: false, + live_output_toggle_enabled: false, elapsed_running: Duration::ZERO, last_resume_at: Instant::now(), is_paused: false, @@ -69,10 +78,20 @@ impl StatusIndicatorWidget { // + optional ellipsis line per truncated message + 1 spacer line let inner_width = width.max(1) as usize; let mut total: u16 = 1; // status line + let text_width = inner_width.saturating_sub(3); // account for " ↳ " prefix + if self.live_line.is_some() { + total = total.saturating_add(1); // blank line before live preview + if text_width > 0 { + let wrapped = + textwrap::wrap(self.live_line.as_deref().unwrap_or_default(), text_width); + total = total.saturating_add(wrapped.len() as u16); + } else { + total = total.saturating_add(1); + } + } if !self.queued_messages.is_empty() { total = total.saturating_add(1); // blank line between status and queued messages } - let text_width = inner_width.saturating_sub(3); // account for " ↳ " prefix if text_width > 0 { for q in &self.queued_messages { let wrapped = textwrap::wrap(q, text_width); @@ -89,6 +108,9 @@ impl StatusIndicatorWidget { // At least one line per message if width is extremely narrow total = total.saturating_add(self.queued_messages.len() as u16); } + if self.live_output_toggle_enabled { + total = total.saturating_add(1); + } total.saturating_add(1) // spacer line } @@ -110,6 +132,30 @@ impl StatusIndicatorWidget { self.frame_requester.schedule_frame(); } + pub(crate) fn set_live_output_preview(&mut self, preview: Option) { + if self.live_line == preview { + return; + } + self.live_line = preview; + self.frame_requester.schedule_frame(); + } + + pub(crate) fn set_live_output_visible(&mut self, visible: bool) { + if self.live_output_visible == visible { + return; + } + self.live_output_visible = visible; + self.frame_requester.schedule_frame(); + } + + pub(crate) fn set_live_output_toggle_enabled(&mut self, enabled: bool) { + if self.live_output_toggle_enabled == enabled { + return; + } + self.live_output_toggle_enabled = enabled; + self.frame_requester.schedule_frame(); + } + pub(crate) fn pause_timer(&mut self) { self.pause_timer_at(Instant::now()); } @@ -180,11 +226,25 @@ impl WidgetRef for StatusIndicatorWidget { // Build lines: status, then queued messages, then spacer. let mut lines: Vec> = Vec::new(); lines.push(Line::from(spans)); + let text_width = area.width.saturating_sub(3); // " ↳ " prefix + if self.live_line.is_some() { + lines.push(Line::from("")); + } if !self.queued_messages.is_empty() { lines.push(Line::from("")); } + if let Some(live) = &self.live_line { + if text_width > 0 { + for (i, piece) in textwrap::wrap(live, text_width as usize).iter().enumerate() { + let prefix = if i == 0 { " ↳ " } else { " " }; + let content = format!("{prefix}{piece}"); + lines.push(Line::from(content.dim().italic())); + } + } else { + lines.push(Line::from(" ↳".dim().italic())); + } + } // Wrap queued messages using textwrap and show up to the first 3 lines per message. - let text_width = area.width.saturating_sub(3); // " ↳ " prefix for q in &self.queued_messages { let wrapped = textwrap::wrap(q, text_width as usize); for (i, piece) in wrapped.iter().take(3).enumerate() { @@ -206,6 +266,21 @@ impl WidgetRef for StatusIndicatorWidget { .dim(), ); } + if self.live_output_toggle_enabled { + let action = if self.live_output_visible { + " hide output" + } else { + " show output" + }; + lines.push( + Line::from(vec![ + " ".into(), + key_hint::ctrl(KeyCode::Char('r')).into(), + action.into(), + ]) + .dim(), + ); + } let paragraph = Paragraph::new(lines); paragraph.render_ref(area, buf); diff --git a/scripts/count_to_30.rs b/scripts/count_to_30.rs new file mode 100755 index 0000000000..93db87bf6f --- /dev/null +++ b/scripts/count_to_30.rs @@ -0,0 +1,24 @@ +#!/usr/bin/env -S rust-script +//! Stream incremental counter output once per second up to 30. + +use std::io::{self, Write}; +use std::thread::sleep; +use std::time::Duration; + +const LETTERS: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"; + +fn main() { + for value in 0..=30 { + println!("{value}"); + // Ensure the line is flushed immediately so the TUI can display it. + io::stdout().flush().expect("flush stdout"); + if let Some(&letter) = LETTERS.get(value as usize) { + let mut stderr = io::stderr(); + write!(stderr, "{}\n", letter as char).expect("write stderr"); + stderr.flush().expect("flush stderr"); + } + if value < 30 { + sleep(Duration::from_secs(1)); + } + } +}