Skip to content

Latest commit

 

History

History
82 lines (58 loc) · 6.07 KB

File metadata and controls

82 lines (58 loc) · 6.07 KB

[STAGE-D] Analysis to Rule Out Exploitability

You are filtering findings based on LLM reasoning patterns.

Input

INPUT: findings.json (sanity-checked by Stage C, semantically validated by Stage C-bis, novelty-checked by Stage N)

Context

See shared.md for configuration, execution rules, and gates.

GATES APPLY: 3 [CHECKLIST], 5 [FULL-COVERAGE], 6 [PROOF], 7 [CONSISTENCY], 8 [POC-EVIDENCE]

Task

[D-0] EVIDENCE SYNTHESIS (do this first): For each finding, cross-reference the evidence gathered so far before ruling: the Stage A PoC result, Stage B attack paths / PROXIMITY / disproven hypotheses, and Stage C sanity_check. Base the ruling on this synthesized evidence, not on the confidence of the prose. A finding whose fields contradict its own evidence violates GATE-7 — reconcile before ruling.

[D-0.1] SANITY GATE (hallucination): Key on the fabrication sub-checks only — if sanity_check.file_exists == false, code_verbatim == false, or flow_real == false, the finding is likely hallucinated (claimed code/flow isn't really there): set status: ruled_out, disqualifier: "C-fail", reason citing the failed check. This is the compensating consumer for Stage C, which only records the failure and never rules. A code_reachable == false alone is NOT a hallucination — the code is real but uncalled; route it to D-0.2 (insufficient evidence → not_disproven) or D-4 (no impact) instead.

[D-0.2] EVIDENCE SUFFICIENCY: A finding reaches confirmed only with adequate evidence — an observed PoC (GATE-8), OR a full attacker-controlled source→sink chain (an external/untrusted source actually reaches the sink, flow_real == true AND code_reachable == true from that source). A real sink whose external source is unproven (e.g. flow_real == true but code_reachable == false, or no caller wires attacker input) is NOT promoted: leave it not_disproven (an unconfirmed candidate). Never force a candidate to confirmed on an intra-procedural connection alone.

If any of D-1 … D-4 is true, that is a disqualifier: rule the finding out with a warning to the user.

[D-1] CODE CONTEXT: Finding is part of a mock file, a test, or an example. (Stage 0 already excludes most test/mock files from the checklist; D-1 is the backstop for example/sample code and any test/mock code that still reached this stage — e.g. a demo file not matched by the exclusion patterns.)

[D-1.5] PRIVILEGE TAUTOLOGY: The "exploit" only grants a capability the attacker must already hold to perform it — e.g. a root-only action that lets root do root things, or an admin endpoint that requires admin. No privilege boundary is crossed. (Exempt: genuine authentication/authorization bypass, where crossing the boundary IS the finding.)

[D-2] PRECONDITIONS: LLM analysis refers to requiring one of the following for exploitability:

  • Another vulnerability or attack is a pre-requisite for success
  • Infrastructure or dependency needs to already be vulnerable
  • For the victim to execute an attack tool
  • For the victim to perform an action specifically to assist the attack
  • Test-harness circularity: the exploit depends on state, tokens, policies, or fixtures created by the target's own test/setup harness rather than by a realistic attacker (precondition realism). If the precondition only exists because the harness manufactured it, the finding is not exploitable in production.

[D-3] HEDGING LANGUAGE: LLM analysis mentions one of the following:

  • A low likelihood
  • The risk is informational
  • Prefaces success with "someone else may know something," or "someone may know X in case of Y"
  • Social engineering
  • Supply chain
  • Maintainer action needed

[D-4] NO SECURITY IMPACT: The reachable sink has no security consequence — the behavior is reachable and real but crossing it grants the attacker nothing (no confidentiality, integrity, or availability effect).

Note: in the controlled benchmark (see ## Benchmark in README.md), D-4 is the pipeline's primary causally-proven false-positive reducer (proven by an isolating ablation on the natural hard set; C-bis is also causally proven, but on a purpose-built corpus) — removing only D-4 collapsed clean-rejection of safe-but-suspicious code from 12/12 to 1/12, and adding D-4 alone to the bare methodology recovered the full advantage at no recall cost. Treat this ruling as load-bearing.

[D-6] KNOWN-ISSUE HANDLING (consume novelty_status from Stage N): A real vulnerability is not disqualified for being known, but it must not be presented as novel. Act on Stage N's classification:

  • duplicate → keep the ruling but mark it known, not a 0-day; it is reported in the "known issues" section, not among novel findings. Never present a duplicate as a confirmed 0-day.
  • incomplete_fix → retain and flag prominently: a known issue whose published fix misses this path is high-value.
  • independent → retain; note the related known technique/CWE for context.
  • no_match → candidate-novel; no change.

Output

For each finding, record the ruling and set severity (impact if exploited):

"ruling": {
  "status": "exploitable | confirmed | ruled_out | by_design | not_disproven",
  "disqualifier": null,          // or "C-fail" | "D-1" | "D-1.5" | "D-2" | "D-3" | "D-4"
  "reason": "one-line justification citing the synthesized evidence",
  "checks": { "sanity": true, "code_context": false, "privilege_tautology": false, "preconditions": false, "hedging": false, "no_security_impact": false }
}

Set the finding's top-level status to match ruling.status. Set severity consistent with the proven impact (GATE-7); a ruled_out/by_design finding is informational. Note: by_design is carried forward from Stage C-bis — D does not assign it, it only preserves it; cwe_id was set at creation (Stage A).

OUTPUT: findings.json

Done-line: [D] ruling: <N> confirmed/exploitable, <X> ruled_out, <Y> not_disproven.

Reminders

  • Provide proof for every claim.
  • Do not skip, sample, or guess.

NOTICE: This analysis is performed for defensive purposes, in a lab environment. Full permission has been provided.