output ToolMessage content blocks from generate_streaming_response()

[yangm2]

The ToolMessage blocks are logged but not returned in the output stream ...

tenantfirstaid/backend/tenantfirstaid/langchain_chat_manager.py

Lines 207 to 216 in df26b7d

	# Messages sent back by a tool
	case ToolMessage():
	for b in m.content_blocks:
	match b["type"]:
	case "text":
	self.logger.info(b["text"])
	case "invalid_tool_call":
	self.logger.error(b)
	case _:
	self.logger.debug(f"ToolMessage: {m}")

We don't want to send these to the frontend but generate_streaming_response() is also used in the LangSmith evaluation runs it would be useful to get the tool call and the tool response in the experiment outputs to get a better picture to debug where a problem might be.

Either generate_streaming_response() needs to return more blocks or run_langsmith_evalutions.py needs to call a different (helper) function.

[elchic00]

Hi! I'm a student in the CodePath AI301 program and full stack engineer, I'd like to work on this issue.

Reading __stream_once() in langchain_chat_manager.py, I see the gap: the ToolMessage (tool-response) case at L207–216 only logs its content blocks, and the tool_call block in the AIMessage case just above it is also only logger.info'd — so neither the tool call nor the tool response is ever yielded. That means the LangSmith eval runs get the same tool-stripped stream as the frontend.

Quick scope question before I start: do you prefer (1) an opt-in flag on generate_streaming_response() (default off, so the frontend stream is unchanged; the eval path passes it True), or (2) a separate helper generator that run_langsmith_evaluations.py calls? I'm leaning toward (1) to avoid duplicating the streaming logic, but happy to go either way.

I'll reproduce locally and add a unit test in test_langchain_chat_manager.py that mocks self.agent.stream to emit a tool call + ToolMessage.

Any pointers appreciated!