Skip to content

Latest commit

 

History

History
140 lines (78 loc) · 12.7 KB

File metadata and controls

140 lines (78 loc) · 12.7 KB

Cascade Identification

If X changes, what downstream breaks — trace the impact before it happens.


What This Skill Does

Every change in an agentic system cascades. A framework update affects every agent that loads it. A data source change affects every agent that queries it. An upstream agent's output format change affects every downstream agent that reads it. A shift in business priorities affects every framework that encodes those priorities.

Most teams discover cascades after the change. Something starts behaving unexpectedly. Someone traces it back through three layers of dependencies and finds that a "small update" to a reasoning framework two weeks ago shifted the behaviour of an agent four hops downstream. The change looked safe. The cascade wasn't visible until its effects accumulated.

This is preventable. Not by avoiding change — agentic systems must evolve. But by tracing the impact before you make the change, so you know what to test, what to monitor, and what to be concerned about. Or, if something has already shifted and you're diagnosing why, by tracing backward from the symptom to the change that caused it.


Why Cascades Are Dangerous

A cascade isn't just "A affects B which affects C." It's the way the effect transforms as it propagates.

You update a reasoning framework's confidence thresholds. The direct effect is clear: the scoring agent that loads this framework will produce different confidence levels. Scores that were 80% might now be 72%. You test the scoring agent. It works correctly — the new thresholds are more accurate, which is why you made the change.

But the shaping agent downstream treats anything above 70% as high-confidence. Under the old calibration, most strong signals cleared 80%, well above the threshold. Under the new calibration, some of those same signals land at 72% — still above the threshold, but barely. The shaping agent's behaviour hasn't technically changed (72% > 70% = high confidence), but the margin has evaporated. Signals that used to clear the bar comfortably now barely clear it, and signals that were borderline now fall below.

The generation agent downstream of shaping receives fewer high-confidence signals. It starts producing more hedged, cautious content. Not wrong — appropriate for lower confidence. But the confidence hasn't actually dropped in the real world. The scoring recalibration shifted the numbers, the shaping agent passed the shift through, and the generation agent responded to a confidence drop that was mathematical, not substantive.

The validation agent sees the hedged content and compares it to historical outputs. The content looks less confident than usual. It flags it for review. The review volume spikes. The human reviewers start rubber-stamping because most flagged items are fine — just slightly more hedged. They develop review fatigue. And then, when a genuinely problematic output appears in the review queue, it gets rubber-stamped too.

One framework threshold change. Five agents affected. The final effect — a problematic output getting approved — looks nothing like the original change. Nobody would connect "we adjusted confidence thresholds" to "a bad output got through review" without tracing the entire cascade.


How to Trace a Cascade

1. Direct Impact

What components are immediately affected by this change?

Start with the change itself and ask what touches it directly:

  • Framework change → every agent that loads this framework, at any depth
  • Data source change → every agent that queries this source, on any field
  • Agent output change → every downstream agent that reads this output
  • Data model change → every agent that reads or writes to the affected tables
  • Business rule change → every framework that encodes this rule, and every agent that loads those frameworks
  • Shared store change → every agent that reads from this store

Your dependency map (from the Dependency Mapping skill) is the starting point. Each filled cell in the dependency matrix that intersects with the changed component is a direct impact.

2. Indirect Impact

What components are affected by the components that were directly affected?

This is where cascades become non-obvious. Follow each directly affected component and ask: who depends on this component's output?

A scoring framework update affects the scoring agent (direct). The scoring agent's output feeds the shaping agent (indirect first-order). The shaping agent's output feeds the generation agent (indirect second-order). The generation agent's output feeds the validation agent (indirect third-order). The validation agent's results feed the trust tracker (indirect fourth-order).

Trace the cascade until you reach the edges of the system — agents that don't pass their output to anyone else, or data stores that are terminal. The cascade ends when it reaches a component with no downstream consumers.

At each hop, ask: does this component transform the effect, amplify it, dampen it, or pass it through unchanged?

  • Transform: The shaping agent converts a confidence score into a content strategy decision. The effect changes character — it's no longer about numbers, it's about how the content is structured.
  • Amplify: The validation agent flags everything below a threshold. A small score shift can cause a large change in the number of flagged items. The effect grows.
  • Dampen: An agent that uses the upstream output as one input among many. The effect is diluted by the other inputs. The cascade weakens at this point.
  • Pass through: An agent that forwards the upstream output without interpreting it. The effect arrives at the next hop unchanged.

Amplification points are the most dangerous — they turn small changes into large effects. A 5% shift in confidence scores might cause a 40% increase in review volume if the validation threshold sits at the right spot in the distribution.

3. Silent Impact

What components might be affected in ways that aren't immediately visible?

These are the cascades that don't show up as errors, format mismatches, or output changes. They show up as subtle shifts in quality, relevance, or accuracy that take weeks to notice.

Stale learned adjustments. A learned adjustment references categories from the old framework version. The adjustment is still active — it passed through the last decay check — but it's now referencing categories that don't exist or mean something different. The agent applies the adjustment, but it targets the wrong thing. The output shifts, but no error is raised.

Misaligned baselines. A baseline was established under the old framework. It's still being used for comparison, but it no longer represents what "normal" looks like under the new framework. Deviation detection produces false positives (if the baseline is too far from the new normal) or false negatives (if the baseline has been informally adjusted to compensate).

Orphaned conflict resolution. A conflict resolution rule assumes a specific output format from the changed agent. The format changed, and now the rule encounters data it can't resolve. Rather than erroring, the rule falls back to a default — which may not be the right resolution for the new format.

Historical data mismatch. Historical data in shared stores was produced under the old framework. Current agents read this data alongside current data, assuming consistent interpretation. The historical data was scored differently, categorised differently, or calibrated differently — but it carries no marker distinguishing it from current data. The agent blends old and new interpretations without knowing it.

4. Impact Severity

For each affected component, assess the type of impact:

Hard failure. The agent can't proceed because it expects something that's no longer there. A required field is missing. An expected format doesn't parse. A dependency is unavailable. These are caught immediately — the system errors, someone investigates, the cascade is visible.

Silent failure. The agent runs but its reasoning is based on outdated assumptions. This is the most dangerous category. The agent produces output that looks normal. The output is wrong in ways that won't be caught until the effects accumulate. Nobody investigates because nobody sees a problem.

Degraded output. The agent works but misses something the change introduced. It doesn't use the new field. It doesn't recognise the new signal type. It doesn't adjust for the new calibration. The output is incomplete rather than wrong — correct as far as it goes, but missing something that should be there.

Clean boundary. The change doesn't cross the dependency boundary. The agent is genuinely unaffected. These are as important to identify as the affected components — they tell you what you don't need to test, which saves effort and focus.

Silent failures are the priority. Hard failures take care of themselves. Degraded output is visible if you're looking for it. Silent failures compound until the entire system has drifted.

5. Mitigation

For each cascade path, determine how to manage the risk:

Incremental deployment. Update one component, verify its output, then update the next. Follow the cascade path from source to edge. At each step, verify that the output matches expectations before proceeding to the next hop. This is slow but safe.

Atomic deployment. Update all affected components simultaneously. This avoids the intermediate state where some components are on the new version and others are on the old. But it requires identifying every affected component correctly — if you miss one, it's running against a world that changed without it.

Rollback path. If the change causes unexpected cascades, can you revert? Framework versioning, baseline snapshots, adjustment archives — these make rollback possible. Without them, you're forward-only, and forward-only means every cascade must be resolved in place.

Monitoring gates. What would you look at to know whether the cascade caused problems? Output quality metrics at each affected agent. Confidence distributions. Review volume. User feedback. Define these before the change, not after — because after, you're investigating a problem, and hindsight monitoring misses the subtle shifts that preceded the visible failure.


Common Cascade Patterns

The Framework Cascade. A reasoning framework update changes how agents interpret data. Every agent loading that framework now produces different outputs. Every downstream agent receiving those outputs behaves differently. The entire pipeline shifts. This is the most common cascade and the most predictable — your dependency map shows exactly which agents are affected.

The Data Source Cascade. A schema change in an external system adds a new field, renames an existing one, or changes the value set of a controlled field. The data reading guide doesn't mention the change. One agent discovers the new data and starts using it. Other agents don't know it exists. Inconsistent data usage across the pipeline — some agents reason with the new information and some don't.

The Learning Cascade. A new human reviewer with different preferences starts editing outputs. The learning system observes different patterns. Soft adjustments shift to match the new reviewer's preferences. Pipeline behaviour changes. The system is now aligned to a different person's judgment without anyone deciding it should be. This cascade is especially dangerous because it happens gradually and looks like healthy adaptation.

The Decay Cascade. The lifecycle agent adjusts its decay schedule. Signals that used to age out after 60 days now persist for 90. Agents downstream now reason against older intelligence they weren't designed to handle. Their outputs shift. Learned adjustments based on those outputs shift. The whole system drifts because the janitor changed its schedule, and nobody traced the downstream effects of keeping data around longer.


References

This skill loads:

  • references/cascade-tracing.md — systematic approach to tracing change impact through a system
  • references/silent-failure-patterns.md — how cascades manifest as degradation rather than breakage

This Completes the Agent Drift Prevention Package

You now have:

  1. Dependency Mapping — what's connected to what
  2. Decay Profiling — what type of decay fits each knowledge type
  3. Learning Cycle Design — how the system improves per domain
  4. Drift Surface — where scope definitions go stale
  5. Cascade Identification — what breaks when things change

To design your agents from scratch, use Agent System Design. To encode institutional knowledge, use Agent Knowledge Curator. To challenge your assumptions about an existing system, try Agent Edge Cases.