Skip to content

Commit 470e4eb

Browse files
πŸ€– perf: fix streaming content delay from ORPC schema validation (#774)
## Problem After the ORPC migration (commit `3ee72886`), streaming message content appeared delayed: - βœ… Token count & speed updates live - βœ… Message item (model name, time) appears immediately - ❌ Actual message content is delayed by several seconds ## Root Cause: Zod Union Validation Order ORPC validates every yielded event against `WorkspaceChatMessageSchema`. The schema was defined as: ```typescript export const WorkspaceChatMessageSchema = z.union([ MuxMessageSchema, // ← Tried FIRST for every event z.discriminatedUnion("type", [...]), // ← Streaming events live here ... ]); ``` **The problem:** `z.union()` tries each schema in order until one passes. For every `stream-delta` event (20+ per second during streaming): 1. `MuxMessageSchema` is tried first β†’ **fails** (wrong shape: has `id`/`role`/`parts`, stream-delta has `type`/`messageId`/`delta`) 2. Then `discriminatedUnion` is checked β†’ finds matching `type: "stream-delta"` β†’ passes This failed validation attempt on `MuxMessageSchema` runs for **every single streaming event**. With rapid deltas, the overhead accumulates and causes visible delay. ### Why Token Count Updated But Content Didn't Both depend on the same event flow, but token counts are accumulated totals that remain stable once computed. Message content requires constructing new `DisplayedMessage` objects with accumulated text. The validation overhead delayed the entire pipeline, but cumulative metrics (tokens) appeared more responsive than the content itself. ## Solution Reorder the union to put `discriminatedUnion` first: ```typescript export const WorkspaceChatMessageSchema = z.union([ z.discriminatedUnion("type", [...]), // Most streaming events match here instantly (O(1) lookup) WorkspaceInitEventSchema, MuxMessageSchema, // Full messages only on stream-end/history replay ]); ``` Since streaming events have a `type` field, `discriminatedUnion` provides O(1) lookup - no failed validation attempts. Also consolidated the two separate `discriminatedUnion` calls into one for cleaner code. ## Changes - `src/common/orpc/schemas/stream.ts`: Reordered union members, merged discriminated unions, added explanatory comment --- _Generated with `mux`_
1 parent b437977 commit 470e4eb

File tree

1 file changed

+7
-2
lines changed

1 file changed

+7
-2
lines changed

β€Žsrc/common/orpc/schemas/stream.tsβ€Ž

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -225,8 +225,11 @@ export const RestoreToInputEventSchema = z.object({
225225
imageParts: z.array(ImagePartSchema).optional(),
226226
});
227227

228+
// Order matters: z.union() tries schemas in order until one passes.
229+
// Put discriminatedUnion first since streaming events (stream-delta, etc.)
230+
// are most frequent and have a `type` field for O(1) lookup.
231+
// MuxMessageSchema lacks `type`, so trying it first caused validation overhead.
228232
export const WorkspaceChatMessageSchema = z.union([
229-
MuxMessageSchema,
230233
z.discriminatedUnion("type", [
231234
CaughtUpMessageSchema,
232235
StreamErrorMessageSchema,
@@ -241,9 +244,11 @@ export const WorkspaceChatMessageSchema = z.union([
241244
ReasoningDeltaEventSchema,
242245
ReasoningEndEventSchema,
243246
UsageDeltaEventSchema,
247+
QueuedMessageChangedEventSchema,
248+
RestoreToInputEventSchema,
244249
]),
245250
WorkspaceInitEventSchema,
246-
z.discriminatedUnion("type", [QueuedMessageChangedEventSchema, RestoreToInputEventSchema]),
251+
MuxMessageSchema,
247252
]);
248253

249254
// Update Status

0 commit comments

Comments
Β (0)