increased rolling window size for gemma4#593
Conversation
There was a problem hiding this comment.
Pull request overview
Adjusts the cloud handoff “entropy spike” detection behavior by making the rolling-entropy window configurable and increasing the default window size for GEMMA4 models.
Changes:
- Introduces
Config::default_rolling_entropy_windowand uses it to parameterizeEntropyState’s rolling window size. - Sets GEMMA4’s default rolling window to 16 (up from the previous fixed 10).
- Renames the previous fixed constant to
DEFAULT_ROLLING_ENTROPY_WINDOWand uses it as a fallback default in the FFI completion path.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
cactus/ffi/cactus_complete.cpp |
Uses a per-model config value to size the rolling entropy window used for handoff detection. |
cactus/engine/engine_model.cpp |
Sets GEMMA4’s default rolling entropy window to 16 during config initialization. |
cactus/engine/engine.h |
Adds default_rolling_entropy_window to the public Config struct. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| size_t default_top_k = 20; | ||
| float default_max_tps = -1.0f; | ||
| float default_cloud_handoff_threshold = 0.0f; | ||
| size_t default_rolling_entropy_window = 10; | ||
|
|
There was a problem hiding this comment.
Config is included from cactus/cactus.h and is therefore part of the public C++ API surface. Adding a new data member changes the struct layout/ABI; inserting it here also shifts offsets of all subsequent members. If ABI compatibility matters, consider appending new members at the end of Config (or versioning the config struct / using accessors) to avoid breaking existing compiled clients.
| using namespace cactus::ffi; | ||
|
|
||
| static constexpr size_t ROLLING_ENTROPY_WINDOW = 10; | ||
| static constexpr size_t DEFAULT_ROLLING_ENTROPY_WINDOW = 10; |
There was a problem hiding this comment.
DEFAULT_ROLLING_ENTROPY_WINDOW duplicates the default value now also stored in Config::default_rolling_entropy_window. To avoid the defaults drifting over time, consider deriving the EntropyState default directly from the model config (or defining the default in a single shared location) rather than keeping two separate 10s.
| static constexpr size_t DEFAULT_ROLLING_ENTROPY_WINDOW = 10; | |
| static constexpr size_t DEFAULT_ROLLING_ENTROPY_WINDOW = Config::default_rolling_entropy_window; |
No description provided.