Commit 470e4eb
authored
π€ perf: fix streaming content delay from ORPC schema validation (#774)
## Problem
After the ORPC migration (commit `3ee72886`), streaming message content
appeared delayed:
- β
Token count & speed updates live
- β
Message item (model name, time) appears immediately
- β Actual message content is delayed by several seconds
## Root Cause: Zod Union Validation Order
ORPC validates every yielded event against `WorkspaceChatMessageSchema`.
The schema was defined as:
```typescript
export const WorkspaceChatMessageSchema = z.union([
MuxMessageSchema, // β Tried FIRST for every event
z.discriminatedUnion("type", [...]), // β Streaming events live here
...
]);
```
**The problem:** `z.union()` tries each schema in order until one
passes.
For every `stream-delta` event (20+ per second during streaming):
1. `MuxMessageSchema` is tried first β **fails** (wrong shape: has
`id`/`role`/`parts`, stream-delta has `type`/`messageId`/`delta`)
2. Then `discriminatedUnion` is checked β finds matching `type:
"stream-delta"` β passes
This failed validation attempt on `MuxMessageSchema` runs for **every
single streaming event**. With rapid deltas, the overhead accumulates
and causes visible delay.
### Why Token Count Updated But Content Didn't
Both depend on the same event flow, but token counts are accumulated
totals that remain stable once computed. Message content requires
constructing new `DisplayedMessage` objects with accumulated text. The
validation overhead delayed the entire pipeline, but cumulative metrics
(tokens) appeared more responsive than the content itself.
## Solution
Reorder the union to put `discriminatedUnion` first:
```typescript
export const WorkspaceChatMessageSchema = z.union([
z.discriminatedUnion("type", [...]), // Most streaming events match here instantly (O(1) lookup)
WorkspaceInitEventSchema,
MuxMessageSchema, // Full messages only on stream-end/history replay
]);
```
Since streaming events have a `type` field, `discriminatedUnion`
provides O(1) lookup - no failed validation attempts.
Also consolidated the two separate `discriminatedUnion` calls into one
for cleaner code.
## Changes
- `src/common/orpc/schemas/stream.ts`: Reordered union members, merged
discriminated unions, added explanatory comment
---
_Generated with `mux`_1 parent b437977 commit 470e4eb
1 file changed
+7
-2
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
225 | 225 | | |
226 | 226 | | |
227 | 227 | | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
228 | 232 | | |
229 | | - | |
230 | 233 | | |
231 | 234 | | |
232 | 235 | | |
| |||
241 | 244 | | |
242 | 245 | | |
243 | 246 | | |
| 247 | + | |
| 248 | + | |
244 | 249 | | |
245 | 250 | | |
246 | | - | |
| 251 | + | |
247 | 252 | | |
248 | 253 | | |
249 | 254 | | |
| |||
0 commit comments