Skip to content

Conversation

@enyst
Copy link
Collaborator

@enyst enyst commented Nov 14, 2025

Summary

  • Default visualizer no longer prints a "Reasoning:" section when a model provides no Responses reasoning (summary/content).
  • Avoids confusing cases where a "Reasoning:" heading is shown but contains a tool command or unrelated text.

Changes

  • openhands-sdk/openhands/sdk/event/llm_convertible/action.py
    • Only render Reasoning when Responses reasoning has summary or content.
  • openhands-sdk/openhands/sdk/event/llm_convertible/message.py
    • Same conditional rendering logic for assistant/user message events.
  • tests/sdk/conversation/test_visualizer.py
    • Added assertion that ensures no "Reasoning:" label is printed when Responses reasoning is absent for ActionEvent.

Motivation
Some providers (e.g., OpenAI GPT-5.1) may not return reasoning summaries by default. Previously, the UI would still display a "Reasoning:" header and then show other content (like a tool command), which was confusing. This change keeps the UI clean and only shows a Reasoning block when we actually have textual reasoning summary/content to render.

Notes

  • The change is purely presentational. No API behavior modified.
  • Kept the existing highlighting styles and panel titles intact.

Co-authored-by: openhands [email protected]

@enyst can click here to continue refining the PR


Agent Server images for this PR

GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python amd64, arm64 nikolaik/python-nodejs:python3.12-nodejs22 Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:860b8c1-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-860b8c1-python \
  ghcr.io/openhands/agent-server:860b8c1-python

All tags pushed for this build

ghcr.io/openhands/agent-server:860b8c1-golang-amd64
ghcr.io/openhands/agent-server:860b8c1-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:860b8c1-golang-arm64
ghcr.io/openhands/agent-server:860b8c1-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:860b8c1-java-amd64
ghcr.io/openhands/agent-server:860b8c1-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:860b8c1-java-arm64
ghcr.io/openhands/agent-server:860b8c1-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:860b8c1-python-amd64
ghcr.io/openhands/agent-server:860b8c1-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:860b8c1-python-arm64
ghcr.io/openhands/agent-server:860b8c1-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:860b8c1-golang
ghcr.io/openhands/agent-server:860b8c1-java
ghcr.io/openhands/agent-server:860b8c1-python

About Multi-Architecture Support

  • Each variant tag (e.g., 860b8c1-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., 860b8c1-python-amd64) are also available if needed

… ActionEvent/MessageEvent now only render the Reasoning section when\n summary or content is present.\n- Add unit test to assert no Reasoning label when none provided.\n\nCo-authored-by: openhands <[email protected]>
@github-actions
Copy link
Contributor

github-actions bot commented Nov 14, 2025

Coverage

Coverage Report •
FileStmtsMissCoverMissing
openhands-sdk/openhands/sdk/event
   utils.py201525%19–22, 24–27, 29–31, 33–35, 37
openhands-sdk/openhands/sdk/event/llm_convertible
   action.py54983%75, 79–81, 93, 100–101, 130–131
   message.py631280%57, 75, 82, 86, 92, 95–96, 99, 117–118, 123, 139
TOTAL12038553154% 

@openhands-ai
Copy link

openhands-ai bot commented Nov 14, 2025

Looks like there are a few issues preventing this PR from being merged!

  • GitHub Actions are failing:
    • Agent Server

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #1163 at branch `fix/hide-empty-reasoning-visualizer`

Feel free to include any additional details that might help me get this PR into a better state.

You can manage your notification settings

Copy link
Collaborator Author

enyst commented Nov 14, 2025

OpenHands-GPT-5 here. Ran the reasoning example with openai/gpt-5.1-codex-mini, direct OpenAI (no base_url), reasoning_effort=None. As expected, no 'Reasoning:' section was rendered. Below are consecutive console lines: the metrics line before, the panel with the agent's output, and the metrics line after.

�[33m╰─�[0m�[33m─────────�[0m�[33m Tokens: �[0m�[36m↑ input 0�[0m�[33m • �[0m�[35mcache hit N/A�[0m�[33m • �[0m�[34m↓ output 0�[0m�[33m • �[0m�[32m$ 0.00�[0m�[33m �[0m�[33m──────────�[0m�[33m─╯�[0m

�[34m╭─�[0m�[34m────────────────────────────�[0m�[34m �[0m�[1;34mMessage from Agent�[0m�[34m �[0m�[34m────────────────────────────�[0m�[34m─╮�[0m
�[34m│�[0m                                                                              �[34m│�[0m
�[34m│�[0m hello                                                                        �[34m│�[0m
�[34m│�[0m                                                                              �[34m│�[0m
�[34m╰─�[0m�[34m────�[0m�[34m Tokens: �[0m�[36m↑ input 10.76K�[0m�[34m • �[0m�[35mcache hit 0.00%�[0m�[34m • �[0m�[34m↓ output 14�[0m�[34m • �[0m�[32m$ 0.0027�[0m�[34m �[0m�[34m─────�[0m�[34m─╯�[0m

No 'Reasoning:' label present in the visualization, which is the expected behavior.

Token counters updated between panels (input/output increased).

Copy link
Collaborator Author

enyst commented Nov 14, 2025

OpenHands-GPT-5 update (ANSI-clean). Reposting the snippet without terminal color codes. Ran openai/gpt-5.1-codex-mini via direct OpenAI, reasoning_effort=None. No 'Reasoning:' section appears.

╰────────── Tokens: ↑ input 0 • cache hit N/A • ↓ output 0 • $ 0.00 ───────────╯

╭───────────────────────────── Message from Agent ─────────────────────────────╮
│                                                                              │
│ hello                                                                        │
│                                                                              │
╰───── Tokens: ↑ input 10.76K • cache hit 0.00% • ↓ output 14 • $ 0.0027 ──────╯

Confirmed: no 'Reasoning:' label present; token counters incremented between panels.

@enyst
Copy link
Collaborator Author

enyst commented Nov 14, 2025

OpenHands-GPT-5 follow-up (ANSI-clean): reasoning_effort=high, no summaries. Showing the first moment reasoning tokens appear: the metrics line before, the event in between (no Reasoning: section), and the metrics line after.

╰───── Tokens: ↑ input 44.6K • cache hit 89.55% • ↓ output 170 • $ 0.0025 ─────╯


╭──────────────────────────────── Observation ─────────────────────────────────╮
│                                                                              │
│ Tool: terminal                                                               │
│ Result:                                                                      │
│ <a name="readme-top"></a>                                                    │
│                                                                              │
│ <div align="center">                                                         │
│   <img                                                                       │
│ src="https://raw.githubusercontent.com/OpenHands/docs/main/openhands/static/ │
│ img/logo.png" alt="Logo" width="200">                                        │
│   <h1 align="center">OpenHands Software Agent SDK </h1>                      │
│ </div>                                                                       │
│                                                                              │
│                                                                              │
│ <div align="center">                                                         │
│   <a                                                                         │
│ href="https://github.com/OpenHands/software-agent-sdk/blob/main/LICENSE"><im │
│ g                                                                            │
│ src="https://img.shields.io/github/license/OpenHands/software-agent-sdk?styl │
│ e=for-the-badge&color=blue" alt="MIT License"></a>                           │
│   <a href="https://all-hands.dev/joinslack"><img                             │
│ src="https://img.shields.io/badge/Slack-Join%20Us-red?logo=slack&logoColor=w │
│ hite&style=for-the-badge" alt="Join our Slack community"></a>                │
│   <br>                                                                       │
│   <a href="https://docs.openhands.dev/sdk"><img                              │
│ src="https://img.shields.io/badge/Documentation-000?logo=googledocs&logoColo │
│ r=FFE165&style=for-the-badge" alt="Check out the documentation"></a>         │
│   <a href="https://arxiv.org/abs/2511.03690"><img                            │
│ src="https://img.shields.io/badge/Paper-000?logoColor=FFE165&logo=arxiv&styl │
│ e=for-the-badge" alt="Tech Report"></a>                                      │
│   <a                                                                         │
│ href="https://docs.google.com/spreadsheets/d/1wOUdFCMyY6Nt0AIqF705KN4JKOWgeI │
│ 4wUGUP60krXXs/edit?gid=811504672#gid=811504672"><img                         │
│ src="https://img.shields.io/badge/SWEBench-72.8-000?logoColor=FFE165&style=f │
│ or-the-badge" alt="Benchmark Score"></a>                                     │
│   <br>                                                                       │
│   <!-- Keep these links. Translations will automatically update with the     │
│ README. -->                                                                  │
│   <a                                                                         │
│ href="https://www.readme-i18n.com/OpenHands/software-agent-sdk?lang=de">Deut │
│ sch</a> |                                                                    │
│   <a                                                                         │
│ href="https://www.readme-i18n.com/OpenHands/software-agent-sdk?lang=es">Espa │
│ ñol</a> |                                                                    │
│   <a                                                                         │
│ href="https://www.readme-i18n.com/OpenHands/software-agent-sdk?lang=fr">fran │
│ çais</a> |                                                                   │
│   <a                                                                         │
│ href="https://www.readme-i18n.com/OpenHands/software-agent-sdk?lang=ja">日本 │
│ 語</a> |                                                                     │
│   <a                                                                         │
│ href="https://www.readme-i18n.com/OpenHands/software-agent-sdk?lang=ko">한국 │
│ 어</a> |                                                                     │
│   <a                                                                         │
│ href="https://www.readme-i18n.com/OpenHands/software-agent-sdk?lang=pt">Port │
│ uguês</a> |                                                                  │
│   <a                                                                         │
│ href="https://www.readme-i18n.com/OpenHands/software-agent-sdk?lang=ru">Русс │
│ кий</a> |                                                                    │
│   <a                                                                         │
│ href="https://www.readme-i18n.com/OpenHands/software-agent-sdk?lang=zh">中文 │
│ </a>                                                                         │
│                                                                              │
│   <hr>                                                                       │
│ </div>                                                                       │
│                                                                              │
│ The OpenHands Software Agent SDK is a set of Python and REST APIs for        │
│ **building agents that work with code**.                                     │
│                                                                              │
│ You can use the OpenHands Software Agent SDK for:                            │
│ * One-off tasks, like building a README for your repo                        │
│ * Routine maintenance tasks, like updating dependencies                      │
│ * Major tasks that involve multiple agents, like refactors and rewrites      │
│                                                                              │
│ Importantly, agents can either use the local machine as their workspace, or  │
│ run inside ephemeral workspaces                                              │
│ (e.g. in Docker or Kubernetes) using the Agent Server.                       │
│                                                                              │
│ You can even use the SDK to build new developer experiences: it’s the engine │
│ behind the                                                                   │
│ [OpenHands CLI](https://github.com/OpenHands/OpenHands-CLI) and [OpenHands   │
│ Cloud](https://github.com/OpenHands/OpenHands).                              │
│                                                                              │
│ Get started with some                                                        │
│ [examples](https://docs.openhands.dev/sdk/guides/hello-world) or [check out  │
│ the docs](https://docs.openhands.dev/sdk) to learn more.                     │
│                                                                              │
│ ## Quick Start                                                               │
│                                                                              │
│ Here's what building with the SDK looks like:                                │
│                                                                              │
│ ```python                                                                    │
│ import os                                                                    │
│                                                                              │
│ from openhands.sdk import LLM, Agent, Conversation, Tool                     │
│ from openhands.tools.file_editor import FileEditorTool                       │
│ from openhands.tools.task_tracker import TaskTrackerTool                     │
│ from openhands.tools.terminal import TerminalTool                            │
│                                                                              │
│                                                                              │
│ llm = LLM(                                                                   │
│     model="anthropic/claude-sonnet-4-5-20250929",                            │
│     api_key=os.getenv("LLM_API_KEY"),                                        │
│ )                                                                            │
│                                                                              │
│ agent = Agent(                                                               │
│     llm=llm,                                                                 │
│     tools=[                                                                  │
│         Tool(name=TerminalTool.name),                                        │
│         Tool(name=FileEditorTool.name),                                      │
│         Tool(name=TaskTrackerTool.name),                                     │
│     ],                                                                       │
│ )                                                                            │
│                                                                              │
│ cwd = os.getcwd()                                                            │
│ conversation = Conversation(agent=agent, workspace=cwd)                      │
│                                                                              │
│ conversation.send_message("Write 3 facts about the current project into      │
│ FACTS.txt.")                                                                 │
│ conversation.run()                                                           │
│ print("All done!")                                                           │
│ ```                                                                          │
│                                                                              │
│ For installation instructions and detailed setup, see the [Getting Started   │
│ Guide](https://docs.openhands.dev/sdk/getting-started).                      │
│                                                                              │
│ ## Documentation                                                             │
│                                                                              │
│ For detailed documentation, tutorials, and API reference, visit:             │
│                                                                              │
│ **[https://docs.openhands.dev/sdk](https://docs.openhands.dev/sdk)**         │
│                                                                              │
│ The documentation includes:                                                  │
│ - [Getting Started Guide](https://docs.openhands.dev/sdk/getting-started) -  │
│ Installation and setup                                                       │
│ - [Architecture & Core                                                       │
│ Concepts](https://docs.openhands.dev/sdk/arch/overview) - Agents, tools,     │
│ workspaces, and more                                                         │
│ - [Guides](https://docs.openhands.dev/sdk/guides/hello-world) - Hello World, │
│ custom tools, MCP, skills, and more                                          │
│ - [API                                                                       │
│ Reference](https://docs.openhands.dev/sdk/guides/agent-server/api-reference/ │
│ server-details/alive) - Agent Server REST API documentation                  │
│                                                                              │
│ ## Examples                                                                  │
│                                                                              │
│ The `examples/` directory contains comprehensive usage examples:             │
│                                                                              │
│ - **Standalone SDK** (`examples/01_standalone_sdk/`) - Basic agent usage,    │
│ custom tools, and microagents                                                │
│ - **Remote Agent Server** (`examples/02_remote_agent_server/`) -             │
│ Client-server architecture and WebSocket connections                         │
│ - **GitHub Workflows** (`examples/03_github_workflows/`) - CI/CD integration │
│ and automated workflows                                                      │
│                                                                              │
│ ## Contributing                                                              │
│                                                                              │
│ For development setup, testing, and contribution guidelines, see             │
│ [DEVELOPMENT.md](DEVELOPMENT.md).                                            │
│                                                                              │
│ ## Community                                                                 │
│                                                                              │
│ - [Join Slack](https://openhands.dev/joinslack) - Connect with the OpenHands │
│ community                                                                    │
│ - [GitHub Repository](https://github.com/OpenHands/agent-sdk) - Source code  │
│ and issues                                                                   │
│ - [Documentation](https://docs.openhands.dev/sdk) - Complete documentation   │
│                                                                              │
│ ## Cite                                                                      │
│                                                                              │
│ ```                                                                          │
│ @misc{wang2025openhandssoftwareagentsdk,                                     │
│       title={The OpenHands Software Agent SDK: A Composable and Extensible   │
│ Foundation for Production Agents},                                           │
│       author={Xingyao Wang and Simon Rosenberg and Juan Michelini and Calvin │
│ Smith and Hoang Tran and Engel Nyst and Rohit Malhotra and Xuhui Zhou and    │
│ Valerie Chen and Robert Brennan and Graham Neubig},                          │
│       year={2025},                                                           │
│       eprint={2511.03690},                                                   │
│       archivePrefix={arXiv},                                                 │
│       primaryClass={cs.SE},                                                  │
│       url={https://arxiv.org/abs/2511.03690},                                │
│ }                                                                            │
│ ```                                                                          │
│                                                                              │
│ 📁 Working directory: /workspace/project/software-agent-sdk                  │
│ 🐍 Python interpreter: /openhands/micromamba/envs/openhands/bin/python       │
│ ✅ Exit code: 0                                                              │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯

╭──────────────────────────────── Agent Action ────────────────────────────────╮
│                                                                              │
│ $ cat <<'EOF' > FACTS.txt                                                    │
│ The OpenHands Software Agent SDK offers Python and REST APIs for building    │
│ agents that work with code.                                                  │
│ EOF [timeout: 30.0s]                                                         │
│                                                                              │

╰─ Tokens: ↑ input 59.45K • cache hit 86.13% •  reasoning 256 • ↓ output 534 •─╯

Concise comparison:

  • reasoning_effort=low (no summaries): no “Reasoning:” label and no reasoning counter in tokens (only input/output/cache).
  • reasoning_effort=high (no summaries): tokens lines include a “reasoning …” counter; event panel still shows no “Reasoning:” section (no summaries requested).

@enyst enyst marked this pull request as ready for review November 14, 2025 05:17
content.append(f"- {s}\n")
if has_content:
for b in list(reasoning_item.content or []):
content.append(f"{b}\n")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels that this should be in a utility method to be DRYer

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@OpenHands fix this please

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm on it! enyst can track my progress at all-hands.dev

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of work completed

Goal addressed

  • Reviewer asked to DRY up the duplicated “Reasoning:” rendering logic.

Changes made

  • Added utility
    • File: openhands-sdk/openhands/sdk/event/utils.py
    • Function: render_responses_reasoning_block(reasoning_item, *, leading_newlines=False) -> Text | None
    • Renders a “Reasoning:” block only when Responses reasoning has summary or content; returns None otherwise. Supports optional leading newlines to preserve spacing where needed.
  • Refactored visualizers to use the utility
    • openhands-sdk/openhands/sdk/event/llm_convertible/action.py
      • Use render_responses_reasoning_block(self.responses_reasoning_item)
    • openhands-sdk/openhands/sdk/event/llm_convertible/message.py
      • Use render_responses_reasoning_block(self.llm_message.responses_reasoning_item, leading_newlines=True) to match prior spacing
  • No behavior changes beyond preserving the prior fix to hide “Reasoning:” when absent; code is now simpler and centralized.

Validation performed

  • Ran pre-commit on edited files: ruff format/lint, pycodestyle, pyright all passed.
  • Ran tests for the visualizer: tests/sdk/conversation/test_visualizer.py — 17 passed.
  • Pushed commit to the existing branch fix/hide-empty-reasoning-visualizer:
    • Commit: 1efd2ec
    • Files changed: action.py, message.py, and new utils.py

Checklist

  • Addressed the reviewer’s DRY request: done
  • Kept changes minimal and focused (only 3 files touched): done
  • Preserved existing behavior and presentation semantics: done
  • Pre-commit hooks and relevant tests pass: done

Next steps

  • Ready for review. If desired, we can extend the utility in the future to support additional styling hooks or use it anywhere else that may eventually render Responses reasoning.

View full conversation

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels that this should be in a utility method to be DRYer

I did this, but TBH I think maybe it's not worth it? It appears to repeat itself, but it's a few lines, and they're tricky to split in a very clear way since it's an attribute there. I feel the original is more clear than other options. 🤷

It's also a bit quirky in message.py, because it's not helping understanding and maintainability to hide from sight (aka utility method you need to navigate to) details of what exactly is happening. We had a number of troubles figuring out every detail in this code and it's the kind of code I should know by heart. That may be just me though. Cc: @tofarr

Copy link
Collaborator

@tofarr tofarr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe introduce a utility method as suggested to dry up the code

…ctionEvent/MessageEvent

- Introduce event.utils.render_responses_reasoning_block
- Replace duplicated logic in action/message visualizers

Co-authored-by: openhands <[email protected]>
@github-actions
Copy link
Contributor

[Automatic Post]: It has been a while since there was any activity on this PR. @enyst, are you still working on it? If so, please go ahead, if not then please request review, close it, or request that someone else follow up.

@enyst enyst requested a review from tofarr November 20, 2025 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants