Skip to content

Commit 63188ab

Browse files
l0z4n0-a1claude
andcommitted
ci: harden test_pass2_flag_invokes_executor against env drift
The Pass-2 executor has two valid graceful-degradation paths: missing ANTHROPIC_API_KEY and missing 'anthropic' SDK. The test asserted only on the API key reason, so a fresh CI runner without the SDK installed (the realistic operator install path — SDK is optional) failed the test. Two-pronged fix: - tests: assert the contract (executed=False + non-empty reason referencing SDK or API key) instead of one specific reason string. - ci: install 'anthropic' SDK so the realistic Pass-2 path is exercised. Restores 212/212 on Ubuntu CI. Co-Authored-By: Claude <noreply@anthropic.com>
1 parent 1723453 commit 63188ab

2 files changed

Lines changed: 10 additions & 7 deletions

File tree

.github/workflows/ci.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ jobs:
2424
- name: Install dependencies
2525
run: |
2626
python -m pip install --upgrade pip
27-
pip install pyyaml pytest
27+
pip install pyyaml pytest anthropic
2828
2929
- name: Model currency guard
3030
run: python scripts/check_model_currency.py

tests/test_orchestrator.py

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -68,14 +68,17 @@ def test_no_exceptions_flag_changes_score():
6868

6969

7070
def test_pass2_flag_invokes_executor():
71-
"""--pass2 must populate bundle['pass2'] (executed=True with API key, False without)."""
71+
"""--pass2 must populate bundle['pass2'] (executed=True with API key + SDK, False otherwise)."""
7272
bundle = _run([str(SELF_SKILL), "--target", "opus-4-7", "--pass2"])
7373
assert bundle.get("pass2") is not None, "pass2 result should be present when --pass2 is set"
74-
has_key = bool(os.environ.get("ANTHROPIC_API_KEY"))
75-
if not has_key:
76-
assert bundle["pass2"]["executed"] is False, \
77-
"without API key, pass2 must degrade gracefully"
78-
assert "ANTHROPIC_API_KEY" in bundle["pass2"]["reason"]
74+
pass2 = bundle["pass2"]
75+
if pass2["executed"] is False:
76+
# Graceful degradation contract: must explain why in `reason`.
77+
# Two valid degradation paths: missing SDK or missing API key.
78+
reason = pass2.get("reason", "")
79+
assert reason, "degradation must include a non-empty reason"
80+
assert ("ANTHROPIC_API_KEY" in reason) or ("anthropic" in reason.lower()), \
81+
f"degradation reason must reference SDK or API key; got: {reason!r}"
7982

8083

8184
def test_pass2_omitted_by_default():

0 commit comments

Comments
 (0)