Skip to content

Commit 98896eb

Browse files
wuliang229copybara-github
authored andcommitted
feat(live): let a live streaming tool send messages to the user directly
A streaming tool used to only talk to the model: every value it yielded came back as a FunctionResponse. Therefore it costs model context and could derail the model's reasoning. An Event with message yielded by a streaming tool is now addressed to the user instead. It is enqueued on the invocation's event queue, so the runner appends it to the session and streams it to the client, and it is never sent over the live model connection. Plain values are still sent to the model as FunctionResponse. A tool can mix and match any number of each, in any order. Runner's run_live also now initializes the invocation's event queue and merges it with the live agent's own event stream. Without a queue, anything running under the live agent that enqueues an event -- a streaming tool, or a node -- fails with "_event_queue is not set". Co-authored-by: Liang Wu <wuliang@google.com> PiperOrigin-RevId: 968533249
1 parent 775c1bd commit 98896eb

6 files changed

Lines changed: 1080 additions & 7 deletions

File tree

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# Streaming Tool Events
2+
3+
**In a streaming tool, `yield Event(message=...)` to talk to the user directly,
4+
and `yield <value>` to give the model a result. Mix and match, in any order.**
5+
6+
## Overview
7+
8+
A streaming tool reports progress to the user while streaming results to the
9+
model, so narrating a long-running tool costs no model turn. Only supported in
10+
streaming (live) agents/api.
11+
12+
## Sample Inputs
13+
14+
- `Help me monitor the stock price for $XYZ stock.`
15+
16+
*The tool tells you directly that it connected to the feed, without going
17+
through the model. The price alerts do go to the model, and it reports them
18+
in its own words.*
19+
20+
- `Stop monitoring $XYZ.`
21+
22+
*The model calls `stop_streaming`, which cancels the background monitor.*
23+
24+
## Graph
25+
26+
```mermaid
27+
graph TD
28+
Agent[streaming_tool_events_agent] -->|calls| Monitor(monitor_stock_price)
29+
Agent -->|calls| Stop(stop_streaming)
30+
```
31+
32+
## How To
33+
34+
Write an `async` generator and put it in `tools`. The yielded type picks the
35+
audience:
36+
37+
```python
38+
async def monitor_stock_price(stock_symbol: str) -> AsyncGenerator[Any, None]:
39+
"""Starts a background monitor for the price of the given stock_symbol."""
40+
yield Event(message=f"Connected to the {stock_symbol} price feed.")
41+
yield f"the price for {stock_symbol} is 300"
42+
yield f"the price for {stock_symbol} is 900"
43+
yield Event(message="That is my last update for now.")
44+
```
45+
46+
Key points:
47+
48+
- **User updates**: yield `Event(message=...)` to send a message straight to
49+
the client. `message` takes a string, a `types.Part` or a `types.Content`.
50+
Framework metadata (`author`, `branch`, `invocation_id`, the content role)
51+
is filled in for you; any other field you set on the event is ignored with a
52+
warning, and the message is still delivered.
53+
- **Model results**: yield a plain value (`str`, `dict`, ...) to send a
54+
`FunctionResponse` back to the model.
55+
- **Side effects**: use `tool_context.actions`, not the event.
56+
57+
### Where the message goes
58+
59+
The message is streamed to your client and appended to the session. It does
60+
not go over the live connection, so it consumes no model turns or tokens
61+
during the active turn and cannot derail the model's reasoning mid-task. It is
62+
ordinary session history, though, so the model does see it once the history is
63+
replayed on the next connect.
64+
65+
## Related Guides
66+
67+
- [Event and NodeInfo](../../../../docs/guides/events/event/index.md) - How
68+
`Event` carries content, actions and metadata, including the `message` field
69+
used here.
70+
- [live_bidi_streaming_tools_agent](../live_bidi_streaming_tools_agent/readme.md) -
71+
The streaming tool basics this sample builds on, including `input_stream` and
72+
`stop_streaming`.
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Copyright 2026 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
from . import agent
Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# Copyright 2026 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
import asyncio
16+
from typing import Any
17+
from typing import AsyncGenerator
18+
19+
from google.adk import Event
20+
from google.adk.agents.llm_agent import Agent
21+
from google.adk.tools.function_tool import FunctionTool
22+
23+
24+
async def monitor_stock_price(stock_symbol: str) -> AsyncGenerator[Any, None]:
25+
"""Starts a background monitor for the price of the given stock_symbol.
26+
27+
Call this function ONLY ONCE to initiate monitoring. Once started, it runs
28+
continuously in the background and automatically streams price alerts.
29+
30+
CRITICAL: Do NOT call this function again to "check" or "poll" for updates.
31+
Simply wait for the background task to yield new values and report them.
32+
Calling this again while running will launch a duplicate background task.
33+
"""
34+
print(f"Start monitor stock price for {stock_symbol}!")
35+
36+
# An Event goes straight to the user, without going over the live connection
37+
# to the model, so status chatter costs no model turn and cannot derail the
38+
# model's reasoning mid-task.
39+
yield Event(message=f"Connected to the {stock_symbol} price feed.")
40+
41+
# A plain value goes back to the model as a FunctionResponse, and the model
42+
# decides how to report it. This is what a streaming tool has always done.
43+
await asyncio.sleep(4)
44+
yield f"the price for {stock_symbol} is 300"
45+
46+
yield Event(message="Trading is getting busy, prices are moving fast.")
47+
48+
await asyncio.sleep(4)
49+
yield f"the price for {stock_symbol} is 400"
50+
51+
# Each yield is addressed to exactly one audience, so narrating and
52+
# reporting at the same moment is simply two yields.
53+
await asyncio.sleep(10)
54+
yield f"the price for {stock_symbol} is 900"
55+
yield Event(message=f"That is my last {stock_symbol} update for now.")
56+
57+
58+
# Use this exact function to help ADK stop your streaming tools when requested.
59+
# For example, to stop `monitor_stock_price` the model calls
60+
# stop_streaming(function_name="monitor_stock_price").
61+
def stop_streaming(function_name: str):
62+
"""Stop the streaming.
63+
64+
The body is intentionally empty: ADK intercepts this call and cancels the
65+
named tool's background task itself. Copy it as is.
66+
67+
Args:
68+
function_name: The name of the streaming function to stop.
69+
"""
70+
71+
72+
root_agent = Agent(
73+
# Find supported models in Vertex here: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/live-api
74+
model="gemini-live-2.5-flash-native-audio", # Vertex
75+
# Find supported models in Gemini API here: https://ai.google.dev/gemini-api/docs/models
76+
# model='gemini-2.5-flash-native-audio-preview-12-2025', # Gemini API
77+
name="streaming_tool_events_agent",
78+
instruction="""
79+
You are a monitoring agent. You can monitor a stock price using
80+
monitor_stock_price.
81+
CRITICAL: Only call monitor_stock_price at most once per request. Once
82+
called, it runs continuously in the background. Do NOT call it again to
83+
"poll" or "check" for updates. Simply wait for it to stream a new price
84+
alert to you, and then report that alert to the user.
85+
If you need to stop the monitor, call stop_streaming.
86+
Don't ask too many questions. Don't be too talkative.
87+
""",
88+
tools=[
89+
monitor_stock_price,
90+
FunctionTool(stop_streaming),
91+
],
92+
)

src/google/adk/flows/llm_flows/functions.py

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1026,6 +1026,93 @@ async def _background_task() -> None:
10261026
return tel_ctx.function_response_event
10271027

10281028

1029+
_MESSAGE_EVENT_FIELDS = frozenset({'content', 'id', 'timestamp'})
1030+
"""The Event fields a streaming tool's message is built from.
1031+
1032+
``id`` and ``timestamp`` are stamped at construction, so every Event carries
1033+
them and neither says anything about what the tool asked for.
1034+
"""
1035+
1036+
1037+
def _message_content_for_user(
1038+
event: Event, *, tool: BaseTool
1039+
) -> Optional[types.Content]:
1040+
"""Returns the content to deliver, or None if the event has no message.
1041+
1042+
Only the ``content`` field is considered for delivery. All other fields are
1043+
ignored. The role is set to "user", overriding any other value.
1044+
1045+
Args:
1046+
event: The event the tool yielded.
1047+
tool: The tool that yielded it, named in the warning.
1048+
1049+
Returns:
1050+
The content to send to the user, or None if there is nothing to send.
1051+
"""
1052+
problem = None
1053+
if not event.content:
1054+
problem = 'it has no content, so there is nothing to deliver'
1055+
elif event.model_dump(
1056+
exclude=set(_MESSAGE_EVENT_FIELDS),
1057+
exclude_defaults=True,
1058+
# Load-bearing beside exclude_defaults: a field with a custom serializer
1059+
# skips the default comparison, so ``long_running_tool_ids`` reports as
1060+
# set on every event. This reads the raw value instead.
1061+
exclude_none=True,
1062+
# Only the presence of a field is read, so a mistyped value is not worth
1063+
# a warning of its own.
1064+
warnings=False,
1065+
):
1066+
problem = 'it sets fields beyond the message, which are ignored'
1067+
1068+
if problem:
1069+
logger.warning(
1070+
'Streaming tool `%s` yielded an Event that is not a purely'
1071+
' user-facing message: %s. To send a message, use Event(message=...)',
1072+
tool.name,
1073+
problem,
1074+
)
1075+
if not event.content:
1076+
return None
1077+
return event.content.model_copy(deep=True, update={'role': 'user'})
1078+
1079+
1080+
async def _emit_streaming_tool_event(
1081+
event: Event,
1082+
*,
1083+
tool: BaseTool,
1084+
tool_context: ToolContext,
1085+
invocation_context: InvocationContext,
1086+
) -> None:
1087+
"""Streams an Event yielded by a streaming tool to the user.
1088+
1089+
Args:
1090+
event: The event the tool yielded.
1091+
tool: The tool that yielded it, named in the branch and in any warning.
1092+
tool_context: The context of the call, for its function call id.
1093+
invocation_context: The invocation to enqueue on.
1094+
"""
1095+
content = _message_content_for_user(event, tool=tool)
1096+
if content is None:
1097+
return
1098+
# Built fresh rather than copied, so the delivered event carries the message
1099+
# and nothing else, and each delivery gets its own id and timestamp: a tool
1100+
# may hold one Event and yield it twice, and the session orders events and
1101+
# decides what compaction has already summarized by timestamp.
1102+
await invocation_context._enqueue_event(
1103+
Event(
1104+
content=content,
1105+
author=_require_agent_name(invocation_context),
1106+
invocation_id=invocation_context.invocation_id,
1107+
branch=(
1108+
f'{tool.name}@{tool_context.function_call_id}'
1109+
if tool_context.function_call_id
1110+
else tool.name
1111+
),
1112+
)
1113+
)
1114+
1115+
10291116
async def _process_function_live_helper(
10301117
tool: BaseTool,
10311118
tool_context: ToolContext,
@@ -1115,6 +1202,15 @@ async def run_tool_and_update_queue(
11151202
if inspect.isasyncgen(res):
11161203
async with Aclosing(res) as agen:
11171204
async for result in agen:
1205+
if isinstance(result, Event):
1206+
await _emit_streaming_tool_event(
1207+
result,
1208+
tool=tool,
1209+
tool_context=tool_context,
1210+
invocation_context=invocation_context,
1211+
)
1212+
continue
1213+
11181214
updated_content = _build_function_response_content(
11191215
tool, result, tool_context.function_call_id
11201216
)

src/google/adk/runners.py

Lines changed: 73 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1961,6 +1961,9 @@ async def run_live(
19611961
live_request_queue=live_request_queue,
19621962
run_config=run_config,
19631963
)
1964+
# A streaming tool emits its user-facing events here instead of returning
1965+
# them inline; without a queue those enqueues raise.
1966+
invocation_context._event_queue = asyncio.Queue()
19641967

19651968
invocation_context.agent = self._find_agent_to_run(
19661969
invocation_context.session, root_agent
@@ -1975,19 +1978,82 @@ async def execute(ctx: InvocationContext) -> AsyncGenerator[Event, None]:
19751978
yield event
19761979

19771980
async with aclosing(
1978-
_with_caller_context(
1979-
self._exec_with_plugin(
1980-
invocation_context=invocation_context,
1981-
session=invocation_context.session,
1982-
execute_fn=execute,
1983-
is_live_call=True,
1981+
self._merge_live_event_streams(
1982+
invocation_context,
1983+
_with_caller_context(
1984+
self._exec_with_plugin(
1985+
invocation_context=invocation_context,
1986+
session=invocation_context.session,
1987+
execute_fn=execute,
1988+
is_live_call=True,
1989+
),
1990+
caller_ctx,
19841991
),
1985-
caller_ctx,
19861992
)
19871993
) as agen:
19881994
async for event in agen:
19891995
yield event
19901996

1997+
async def _merge_live_event_streams(
1998+
self,
1999+
ic: InvocationContext,
2000+
agent_events: AsyncGenerator[Event, None],
2001+
) -> AsyncGenerator[Event, None]:
2002+
"""Interleaves the live agent's events with events from ``ic._event_queue``.
2003+
2004+
Code running underneath the live agent — a streaming tool, or a node — has
2005+
no way to yield an event back through the agent's own stream, so it
2006+
enqueues on ``ic._event_queue`` instead. Both sources are drained
2007+
concurrently into one queue and surfaced in the order they are produced.
2008+
2009+
Each source keeps its own post-processing: the agent's events are already
2010+
persisted and plugin-processed by ``_exec_with_plugin``, and the queued
2011+
events by ``_consume_event_queue``, so nothing is handled twice.
2012+
"""
2013+
if ic._event_queue is None:
2014+
raise RuntimeError(
2015+
'Live event stream merging requires an initialized event queue.'
2016+
)
2017+
# Bind the queue to a local: the narrowing above does not reach into the
2018+
# nested pumps below.
2019+
event_queue = ic._event_queue
2020+
done_sentinel = object()
2021+
merged: asyncio.Queue[Any] = asyncio.Queue(maxsize=1)
2022+
2023+
async def _pump_agent_events() -> None:
2024+
try:
2025+
async with aclosing(agent_events) as agen:
2026+
async for event in agen:
2027+
await merged.put(event)
2028+
finally:
2029+
# The queue consumer owns the merged sentinel, so end its stream
2030+
# rather than the merged one; that also lets already-enqueued events
2031+
# drain before the merge finishes.
2032+
await event_queue.put((done_sentinel, None))
2033+
2034+
async def _pump_queued_events() -> None:
2035+
try:
2036+
async with aclosing(
2037+
self._consume_event_queue(ic, done_sentinel)
2038+
) as agen:
2039+
async for event in agen:
2040+
await merged.put(event)
2041+
finally:
2042+
await merged.put(done_sentinel)
2043+
2044+
agent_task = asyncio.create_task(_pump_agent_events())
2045+
queue_task = asyncio.create_task(_pump_queued_events())
2046+
try:
2047+
while True:
2048+
event_or_done = await merged.get()
2049+
if event_or_done is done_sentinel:
2050+
break
2051+
yield event_or_done
2052+
finally:
2053+
# _cleanup_root_task re-raises a failure from either pump.
2054+
await self._cleanup_root_task(agent_task, self.agent.name)
2055+
await self._cleanup_root_task(queue_task, self.agent.name)
2056+
19912057
def _find_agent_to_run(
19922058
self, session: Session, root_agent: BaseAgent
19932059
) -> BaseAgent:

0 commit comments

Comments
 (0)