Skip to content

Conversation momentum implementation - #3314

Open
QuMuzafferEge wants to merge 6 commits into
lemonade-sdk:mainfrom
QuMuzafferEge:ege_2956
Open

Conversation momentum implementation#3314
QuMuzafferEge wants to merge 6 commits into
lemonade-sdk:mainfrom
QuMuzafferEge:ege_2956

Conversation

@QuMuzafferEge

@QuMuzafferEge QuMuzafferEge commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds an opt-in routing.momentum policy block to the router engine to fix route flapping in multi-turn conversations. Today min_chars/max_chars only ever measure the latest user turn, so a long coding question correctly escalates to a strong model via min_chars, but the very next short follow-up ("yes, do it") falls straight back to default_model mid-task — paying a real switch cost (model load/eviction, KV re-prefill, provider cache miss) every time.

routing.momentum replaces the raw last-turn byte length with a momentum-filtered "effective length," recomputed fresh from each request's own message history (no server-side session state — the conversation is already resent every turn, same as any OpenAI-style chat client):

json
"routing": {
"momentum": { "enabled": true, "attack": 1.0, "release": 0.3 }
}

An asymmetric EMA over each user turn's UTF-8 byte length: attack (default 1.0) weights a rising turn so a sudden long/complex turn escalates immediately, release (default 0.3) weights a falling turn so a short follow-up only decays the effective length by 30% per turn — keeping the conversation on the escalated route for several turns before de-escalating. attack = release = 1.0 degenerates exactly to today's behavior. Applies only to min_chars/max_chars; absent or enabled: false is behavior-identical to a policy without the block.

Fixes #2956

Scope
This PR addresses one clear issue or change.
I reviewed the full diff myself before submitting.
I removed unrelated local changes.

Testing
Code builds without errors locally.
I tested this change locally.
I described the testing performed below.

Testing details:

Built and tested on Linux (Ubuntu 24.04 container, matching the cpp-unit-tests CI job's recipe: cmake --preset default → cmake --build --preset default --target cpp-ci-tests → ctest -L "^cpp-ci$").

50/52 cpp-ci tests passed, including every test this change touches:
RoutingPolicyMomentumTest (new) — filter arithmetic (rise/fall/tie/degenerate coefficients, IEEE-exactness), CharsCondition/EvalContext::effective_chars wiring, engine-level fold, trace rationale format.
RoutingPolicyParserTest (extended) — momentum block validation (required enabled, (0,1] coefficient bounds, unknown-key rejection) and schema/parser key parity.
RoutingClassifierServicesTest (extended) — build_route_context collecting user_turn_chars across chat messages, legacy prompt, and Responses input forms.
RoutingConformanceCorpusTest — two new golden groups (l1_momentum_effective_chars, l1_momentum_absent_is_noop) plus every pre-existing group re-validated byte-for-byte unchanged, confirming no regression when the block is absent.
2 pre-existing failures (TelemetryHelpersTest, CliRuntimeOverrideTest) are unrelated to this change — neither file is touched by this diff, and both look like environment artifacts of a from-scratch container vs. the pinned CI image (locale/encoding and a streaming-timing sensitivity respectively).
python test/test_schema_lock.py and python -m unittest test.test_routing_fixtures both pass (schema-lock hash refreshed for the additive routing.momentum schema keys and the decision.schema.json rationale-field description update).

Documentation
Documentation is affected and has been updated.

Added a "Momentum (routing.momentum)" section to docs/dev/router-policy.md (field table, formula, worked example, trace format, explicit non-goals), footnoted the min_chars/max_chars schema descriptions and the frozen-semantics table in src/cpp/resources/schemas/README.md to reference the new opt-in mechanism without redefining the existing frozen unit/comparator.

Breaking Changes
This PR does not introduce breaking changes.

Fully additive and opt-in: new RoutePolicy/RouteContext/EvalContext fields (C++-internal), one new optional routing.momentum schema property, and use of the already-optional trace_entry.rationale field. No existing field is redefined, and every code path is gated behind momentum.enabled (default false), so any policy without the block is behavior-identical to before this change.

@jeremyfowers

Copy link
Copy Markdown
Member

Please give this PR a descriptive title before merging.

@QuMuzafferEge QuMuzafferEge changed the title Resolves #2956 Conversation momentum implementation Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Router] Conversation momentum: stateless hysteresis to prevent per-turn route flapping

3 participants