feat/elevenlabs-live: docs

Author: stepacool

website/docs/user-guide/live/live.mdx

@@ -3,7 +3,11 @@ title: "Voice & Realtime Overview"
 sidebarTitle: "Overview"
 ---
 
-`ag2.live` is the AG2 module for building voice-enabled agents. It covers two complementary patterns: a turn-by-turn **STT → Agent → TTS** pipeline built on top of a regular `Agent`, and a full-duplex **`LiveAgent`** that streams audio to and from a provider's realtime API.
+`ag2.live` is the AG2 module for building voice-enabled agents. It covers two complementary patterns:
+
+1. **CascadeTurn/Turn-by-Turn/Pipeline** `STT → Agent → TTS` approach built on top of a regular `Agent`
+
+2. **Realtime/Speech-to-Speech/S2S** `LiveAgent` approach that streams audio to and from a provider's realtime API.
 
 ## When to use which
 
@@ -12,7 +16,7 @@ sidebarTitle: "Overview"
 | Turn-by-turn voice | `Agent` + a transcriber + a TTS observer | ~1–3 s per turn | You already have a text agent and want to add a voice front-end; you need tool execution, middleware, structured output, or any other text-agent feature. |
 | Realtime full-duplex | `LiveAgent` + `OpenAIRealTimeConfig` / `GeminiRealTimeConfig` | <500 ms | You want barge-in, interruption, semantic VAD, or a phone-call-like UX. |
 
-For the turn-by-turn pattern, OpenAI and ElevenLabs are both available and support different halves incrementally — OpenAI streams transcription, ElevenLabs streams synthesis. See [Provider support](/docs/user-guide/live/stt_tts) for the full matrix. Gemini is realtime-only.
+See [Provider support](/docs/user-guide/live/stt_tts) for the matrix of currently supported providers.
 
 !!! note
     Both patterns share the same audio I/O primitives — [`SoundDevicePlayer`](/docs/user-guide/live/stt_tts) and [`SoundDeviceRecorder`](/docs/user-guide/live/stt_tts) — and the same event stream. You can mix observers (TTS, logging, persistence) across both.

website/docs/user-guide/live/stt_tts.mdx

@@ -7,16 +7,12 @@ The STT → Agent → TTS flow turns any existing `Agent` into a voice agent wit
 
 ## Provider support
 
-Both halves are protocol-based, so providers are interchangeable — but they do not all support incremental delivery, and the two current providers are complementary opposites.
+| Provider | STT | Streaming STT | TTS | Streaming TTS | Audio to Audio |
+|---|---|---|---|---|---|
+| OpenAI | ✅ | ✅ | ✅ | — | ✅ |
+| ElevenLabs | ✅ | — | ✅ | ✅ | — |
+| Gemini | — | — | — | — | ✅ |
 
-| Provider | STT | Streaming STT | TTS | Streaming TTS |
-|---|---|---|---|---|
-| OpenAI | `OpenAITranscriber`, `OpenAITranslationTranscriber` | **Yes** — emits `TranscriptionChunkEvent` deltas | `OpenAITTSConfig` | No — one buffered clip |
-| ElevenLabs | `ElevenLabsTranscriber` (Scribe) | No — whole-file upload | `ElevenLabsTTSConfig`, `ElevenLabsStreamingTTSConfig` | **Yes** — via `ElevenLabsStreamingTTSConfig` |
-| Gemini | — | — | — | — |
-
-!!! note
-    **ElevenLabs is currently the only streaming TTS provider**, and **OpenAI is the only streaming STT provider**. Gemini has no standalone STT/TTS config in `ag2.live` — it is realtime-only, via `GeminiRealTimeConfig` and [LiveAgent](/docs/user-guide/live/live_agent).
 
 Mixing providers across the two halves is supported and often the best setup: transcribe with OpenAI for live captions, speak with ElevenLabs for low time-to-first-byte.
 
@@ -104,10 +100,6 @@ pipeline = ElevenLabsTranscriber("scribe_v2").pipe(agent)
 
 `language_code` is only sent when you set it — leave it unset and Scribe auto-detects. `diarize=True` asks for speaker labels.
 
-!!! warning "Scribe is whole-file, so there are no partials"
-    The endpoint takes a **complete** recording, so there is nothing to emit until it answers. `ElevenLabsTranscriber` only ever sends `TranscriptionCompletedEvent` — never `TranscriptionChunkEvent`. Live-caption code written against the OpenAI transcriber's deltas will go quiet if you swap in Scribe.
-
-    It also blocks the whole turn: `VoicePipeline.ask` awaits `transcribe(...)` before the agent starts, so nothing can be spoken during the upload and transcription. On a short turn this often dominates the latency — see [Where the latency goes](#where-the-latency-goes).
 
 ## Text-to-Speech
 
@@ -207,9 +199,6 @@ if __name__ == "__main__":
     asyncio.run(main())
 ```
 
-!!! warning "Pair the streaming config with the streaming observer"
-    `ElevenLabsStreamingTTSConfig` also implements `synthesize(...)`, so it satisfies the `TTSConfig` protocol and *will* work with the plain `TTSObserver` — but `synthesize` is implemented by draining the whole stream into one buffer, which throws away the entire latency advantage. Use `ElevenLabsStreamingTTSObserver` with `ElevenLabsStreamingTTSConfig`, and `TTSObserver` with `ElevenLabsTTSConfig` / `OpenAITTSConfig`.
-
 ### Streaming directly to the speaker
 
 `ElevenLabsStreamingTTSConfig.stream(...)` is usable on its own, outside any agent, when you just want to speak a known string:
@@ -241,20 +230,6 @@ observers=[ElevenLabsStreamingTTSObserver(tts, min_chars=40)]
 
 Lowering it starts the first sentence sooner, at the cost of more requests and more granular sentences.
 
-## Where the latency goes
-
-Streaming TTS removes one source of delay, not all of them. For a single turn, in order:
-
-| Stage | Cost | Streaming TTS helps? |
-|---|---|---|
-| Transcription | With Scribe, the whole clip is uploaded and transcribed before the agent starts | No |
-| Model time-to-first-token | Provider-dependent; large on reasoning models | No |
-| Sentence buffer fill | Until `min_chars` **and** a sentence boundary is reached | No — lower `min_chars` instead |
-| Synthesis | Time-to-first-byte vs. waiting for the full clip | **Yes** — this is the win |
-
-!!! note
-    Synthesis is awaited inline in the observer, which runs inside the model's token loop. While one sentence is being synthesized the next tokens are not being consumed, so a long reply can still develop small gaps between sentences even in streaming mode.
-
 ## Combining STT and TTS
 
 The full round-trip — voice in, voice out — is just both halves wired up at once: pipe the agent through the transcriber, attach a `TTSObserver`, and share a stream with the player.
@@ -305,6 +280,54 @@ if __name__ == "__main__":
 !!! tip
     `player.join()` blocks the main task until the synthesized audio queue has drained. Use it between turns when you want the assistant to finish speaking before the next recording starts — otherwise the recorder will capture the tail of the assistant's voice.
 
+### Mixing providers
+
+This is an example of how one can use `openai` STT and `elevenlabs` TTS together.
+
+```python linenums="1" hl_lines="19 23 27 35"
+import asyncio
+
+from ag2 import Agent, config
+from ag2.events import TranscriptionChunkEvent
+from ag2.live import (
+    ElevenLabsStreamingTTSConfig,
+    ElevenLabsStreamingTTSObserver,
+    OpenAITranscriber,
+    SoundDevicePlayer,
+    SoundDeviceRecorder,
+)
+
+SAMPLE_RATE = 24000  # pcm_24000, what the player and recorder use
+
+agent = Agent(
+    "assistant",
+    prompt="You are a helpful voice assistant. Keep answers to a few sentences.",
+    config=config.OpenAIConfig("gpt-5-mini", streaming=True),
+    observers=[ElevenLabsStreamingTTSObserver(ElevenLabsStreamingTTSConfig("eleven_flash_v2_5"))],
+)
+
+async def main() -> None:
+    pipeline = OpenAITranscriber("gpt-4o-mini-transcribe").pipe(agent)
+    recorder = SoundDeviceRecorder(sample_rate=SAMPLE_RATE)
+
+    # OpenAI streams the transcript back as deltas — print them as live captions.
+    async def on_caption(event: TranscriptionChunkEvent) -> None:
+        print(event.content, end="", flush=True)
+
+    async with SoundDevicePlayer() as player:
+        player.stream.where(TranscriptionChunkEvent).subscribe(on_caption)
+
+        print("Say something...")
+        voice_input = recorder.record(duration=5)
+        reply = await pipeline.ask(voice_input, stream=player.stream)
+        print(f"\n{reply.body}")
+
+if __name__ == "__main__":
+    asyncio.run(main())
+```
+
+Both halves share one stream, so `TranscriptionChunkEvent` (from the transcriber) and `SynthesizedAudioEvent` (from the observer) land in the same place — the player only subscribes to the latter, leaving the captions to your own subscriber.
+
 ## Examples
 
 - `examples/stt_tts.py/stt_tts.py` — the OpenAI round-trip