Skip to content

CHANGELOG.md

Latest commit

 

History

History
669 lines (535 loc) · 30.5 KB

CHANGELOG.md

File metadata and controls

669 lines (535 loc) · 30.5 KB
Raw

Changelog

All notable changes to Copilot Guardian will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[0.2.7] - 2026-02-13

Security/Robustness Hardening + Confidence Guardrails

Added

  • Confidence-gap ambiguity defense in analysis
    • Added confidence_gap computation between top-ranked hypotheses.
    • Added low_confidence_ambiguity flag and reviewer guidance fields in analysis.json.
    • Added operator-facing warning to use --show-reasoning and larger --max-log-chars when ambiguity is detected.
  • Deterministic no-op slop detection
    • Quality guard now fail-closes comment-only/whitespace-only patch additions (NO_GO).
  • Regression tests for critical paths
    • Added tests/analyze_confidence_gap.test.ts.
    • Extended traversal tests to include both Unix-style (../) and Windows-style (..\\) path traversal variants.

Changed

  • Allowlist/path-scope hardening
    • Hardened glob matching and path normalization for safer scope checks.
    • Added traversal-segment rejection to prevent allowlist bypass via path tricks.
  • MCP configuration resilience
    • Improved cross-platform config path resolution (Windows/macOS/Linux) with normalized paths.
    • Added GitHub token resolution fallback via gh auth token when env vars are absent.
  • Version alignment (docs/runtime metadata)
    • Runtime/package/docs bumped to 0.2.7.

Documentation Evidence Update - 2026-02-15

Added

  • Real output evidence package for reviewer verification
    • Added examples/real-output/standard/ with unmodified artifacts copied from .test-output/:
      • patch_options.json
      • fix.*.patch
      • quality_review.*.json
      • copilot.*.raw*.txt
    • Added examples/real-output/abstain/guardian.report.json copied from .test-output-run-abstain/.
    • Added examples/real-output/README.md documenting context, evidence map, and interpretation.

Changed

  • README now highlights real execution evidence
    • Added Real Output Examples badge in the header.
    • Reworked hero section with stronger value hook and immediate quick-run command.
    • Reordered content flow to Demo -> Why This Matters -> Real Output Showcase for faster reviewer conversion.
    • Strengthened Demo section with explicit What happens in this run checklist and direct artifact path.
    • Added Real Output Showcase with fail-closed NO_GO JSON evidence and raw malformed-response link.
    • Expanded patch spectrum table with explicit HIGH/LOW risk and bold GO/NO_GO verdicts.
    • Expanded Output Files table to include direct example links under examples/real-output/.
    • Added explicit references to both standard and abstain evidence paths for auditability.
    • Added stronger Try It Now CTA with issue/docs/examples links and social star badge.
    • Finalized section ordering for reviewer flow: Challenge Submission -> Demo -> Why This Matters -> Real Output Showcase -> Judge Quick Test.

[0.2.6] - 2026-02-12

npm Publish + Packaging Hygiene

Added

  • Published to npm (latest)
    • Package name: copilot-guardian
    • Published version: 0.2.6
    • Install paths:
      • npm i -g copilot-guardian@latest
      • npx copilot-guardian@latest --help

Changed

  • npm package footprint reduced dramatically
    • Introduced files allowlist in package.json to publish runtime essentials only.
    • Excluded large/non-runtime artifacts (tests, source TS, local test outputs, media-heavy docs).
    • Tarball reduced from ~6.1 MB to ~77 KB (npm pack --dry-run verification).
  • README distribution clarity
    • Added npm version badge.
    • Added explicit npm package URL and install/run guidance using @latest.
    • Adjusted badge layout for readability.
  • Version alignment (docs/runtime metadata)
    • README badges updated to 0.2.6 / v0.2.6.
    • CLI runtime version string updated to 0.2.6.
    • CHANGELOG top release synchronized to 0.2.6.

Fixed

  • Security policy documentation alignment
    • Replaced placeholder contact with info@flamehaven.space.
    • Updated supported version range from 1.0.x to 0.2.x.
    • Updated policy metadata date/version for current release line.

[0.2.5] - 2026-02-11

Speed + Stable Demo Mode

Changed

  • Analysis speed tuning
    • Fast mode reduces deep source-context fetch scope (maxSourceFiles lowered).
    • Analysis model timeout is shortened in fast mode.
  • Quality artifact resilience
    • quality_review.<strategy>.json is now persisted even when model output parse fails.
  • README operational guidance
    • Judge quick test now defaults to stable fast profile (--show-options --fast --max-log-chars 20000).
    • Added immediate no-code speed tuning section using COPILOT_TIMEOUT_MS.
    • Clarified that --show-reasoning is optional and slower.

Fixed

  • Fast-path plumbing consistency
    • Wired fast option into evaluation report metadata and markdown output.

[0.2.4] - 2026-02-11

README Clarity + Version/Tag Alignment

Added

  • Dedicated Forced Abstain documentation
    • Added Forced Abstain Policy (NOT PATCHABLE) section in README.
    • Documented abstain trigger classes, generated artifact (abstain.report.json), and operator action.

Changed

  • Submission metadata alignment
    • Updated README release badges/tag references to v0.2.4.
    • Updated submission section heading and quick-verification path label to v0.2.4.
  • Auto-heal docs clarity
    • Explicitly states that NOT PATCHABLE classification skips patch/apply flow.

Fixed

  • Runtime/version consistency
    • Updated CLI runtime version to 0.2.4.
    • Updated package version metadata (package.json, package-lock.json) to 0.2.4.

[0.2.3] - 2026-02-11

Security Hardening + Safe Auto-Heal Controls

Added

  • Forced abstain policy for non-patchable failure classes
    • Added auth/permission/infra classifiers (401/403, token permission denied, rate limits, runner unavailable, service unavailable).
    • Guardian now emits abstain.report.json and skips patch generation for these classes.
  • Secret redaction fail-closed enforcement
    • Added residual secret-pattern detection after redaction.
    • Analysis now aborts when sensitive token patterns remain in logs.
  • Auto-heal branch safety controls
    • New CLI options:
      • --allow-direct-push (explicit unsafe override)
      • --base-branch <name> (PR target branch)
      • --max-retries <n> (bounded CI rerun attempts; default 3)
    • Default behavior is PR-only safe mode using guardian/run-<run_id>-<suffix> branches.

Changed

  • Deterministic guard hard caps
    • Workflow file edits are forced NO_GO (human review required).
    • File deletion patches are forced NO_GO (human review required).
    • Oversized patch footprint is forced NO_GO beyond safe auto-fix threshold.
  • Evaluation harness security reporting
    • Added abstain-aware reporting fields and rates.
    • Added per-case and aggregate security_severity distribution.
  • Auto-heal execution order
    • Safe branch creation now occurs before patch application in PR-only mode.
    • Failed apply in safe mode performs branch cleanup before exiting.
  • README submission positioning and operator guidance
    • Updated release/version tags to v0.2.3.
    • Updated auto-heal narrative to PR-only safe mode.
    • Clarified artifact generation flow and output locations under .copilot-guardian/.

Fixed

  • Evaluation markdown table formatting
    • Corrected case-table separator column count to match expanded security fields.

[0.2.2] - 2026-02-11

TS Gate Hardening + Scenario Verification

Added

  • TS suppression anti-pattern guard
    • Deterministic review now blocks added @ts-ignore, @ts-nocheck, and eslint-disable.
  • Scenario test for suppression-based fake fix
    • Added regression test to ensure suppression-only patches are rejected.
  • Real-world evaluation harness (eval)
    • Added copilot-guardian eval command for multi-run patchability benchmarking.
    • Supports explicit --run-ids, file-based IDs (--run-file), or recent failed runs (--failed-limit).
    • Generates aggregate reports: eval.report.md, eval.report.json, and eval.cases.json.
    • Adds security-oriented metrics: bypass attempt rate, bypass block rate, and security false-GO rate.

Changed

  • Glob scope matching accuracy
    • Fixed ** wildcard handling so tests/**/*.ts and src/**/*.ts correctly match root-level files.
  • Placeholder marker handling
    • Added-line TODO/FIXME/HACK markers are now fail-closed (NO_GO) instead of soft warning.
  • Security bypass detection expansion
    • Deterministic guard now fail-closes on additional bypass patterns:
      • NODE_TLS_REJECT_UNAUTHORIZED=0
      • GIT_SSL_NO_VERIFY=true
      • strict-ssl false / npm config set strict-ssl false
      • curl -k / --insecure
      • || true / set +e
  • README challenge positioning overhaul
    • Added Why This Is a Copilot CLI Challenge Submission and Judge Quick Test (90 seconds) sections near the top.
    • Added Single Test Mode (Clean Run for GIF + Review) with one-file verification flow and direct artifact links.
    • Added a fail-closed deterministic safety-layer diagram and explicit anti-slop CI positioning.
    • Added explicit runtime note: production path is @github/copilot-sdk, CLI fallback is local experimentation only.
    • Added clarification for legacy unknown command "chat" for "copilot" traces to prevent reviewer confusion.
    • Added final GIF insertion slot for submission-final update.
  • Test surface simplification for submission clarity
    • Reduced tests/ to a single primary scenario file: tests/quality_guard_regression_matrix.test.ts.
    • Removed legacy tests/__mocks__ tree and migrated required SDK stub to tests/mocks/copilot-sdk.mock.ts.

Fixed

  • False out-of-scope rejection
    • Balanced patches touching tests/<file>.ts no longer misclassified as out of scope due glob conversion bug.
  • Real lint gate activation
    • Replaced placeholder lint: skipped (MVP) with tsc --noEmit type-check lint gate for CI validity.
  • Submission artifact hygiene
    • Removed legacy guardian-output.txt and optional sidrce_cert.yaml from repository root to reduce reviewer confusion.
  • Version alignment to 0.2.2
    • Updated package.json, package-lock.json, CLI runtime version, README release badges and submission tag.

[0.2.1] - 2026-02-11

Submission Hardening: Independent TS Quality Core

Added

  • Deterministic TypeScript quality guard (internal algorithm)
    • Added local patch review for scope, bypass anti-patterns, intent alignment, and patch footprint.
    • Merges deterministic verdict with model verdict for final GO/NO_GO decision.
    • Keeps the project independent without external SIDRCE/ai-slop-detector pipeline coupling.

Changed

  • Auto-heal patch selection policy
    • run --auto-heal now selects the best GO strategy by risk_level then slop_score, not first GO hit.
    • Dashboard recommendation uses the same ranking policy for consistent operator decisions.
  • README release metadata
    • Updated badges/tag references to v0.2.1.
    • Replaced static test-count badge with CI workflow badge.

Fixed

  • JSON schema validation compatibility
    • Switched validator runtime to Ajv 2020 to match $schema: draft/2020-12.
    • Eliminates false schema warnings that previously forced valid patches to NO_GO.
  • Jest parse/runtime stability
    • Removed direct import.meta usage from util path resolution to avoid CommonJS parse failures in tests.
    • Updated SDK call expectation in async-exec.test.ts for sendAndWait({ prompt, mode }, timeout).
    • Updated analyze test expectation for fetchRunContext(repo, runId, maxLogChars).

[0.2.0] - 2026-02-11

Submission Edition: Adaptive Failure Intelligence

This release upgrades Guardian into a step-aware CI recovery engine for challenge submission quality.

Added

  • Step-aware diagnosis weighting
    • Hypothesis selection now considers failed-step/category compatibility, not confidence only.
  • Dynamic patch allowlist generation
    • patch_plan.allowed_files is expanded from failed step context (test/lint/build/install).
  • Test evidence extraction
    • Failed test files and assertion signals are parsed from logs and injected into analysis context.
  • New CLI option: --max-log-chars
    • Available on run and analyze commands for wider failure evidence coverage.
  • Auto re-diagnosis guidance
    • When patch spectrum is NO_GO for all options, Guardian suggests a re-run with expanded logs.
  • README submission upgrade
    • Added v0.2.0 submission section and release tag badges without removing prior README content.

Changed

  • Version alignment to 0.2.0
    • package.json, package-lock.json, CLI runtime version, README badge/tag, SIDRCE metadata.
  • MCP prompt strategy
    • Prompt now prioritizes failed-step evidence, especially test assertion context.
  • Allowlist enforcement
    • Auto-apply allowlist supports glob patterns for safer and practical scope checks.

Fixed

  • Quality review fail-open risk
    • Malformed quality JSON now returns NO_GO with high risk instead of permissive GO.
  • Schema bypass detection
    • Out-of-range slop_score values are flagged and normalized with forced NO_GO.
  • False-positive source extraction noise
    • Improved file-path boundary matching to reduce URL-derived pseudo files in deep analysis.

[0.1.4] - 2026-02-10

🎨 UI/UX Enhancements + Critical Parser Fix

This release focuses on visual feedback improvements and response handling robustness to make Guardian production-ready for real execution recording.

Added

  • Enhanced Progress Indicators: Step-by-step status messages throughout analysis pipeline

    • MCP configuration check with success confirmation
    • Workflow context retrieval with metadata display (SHA, workflow path)
    • Deep intelligence source context extraction with file listings
    • Real-time Copilot communication status
    • Artifact creation logs (file-by-file confirmation)

  • Improved Hypothesis Display: Multi-hypothesis visualization enhancements

    • [SELECTED] marker for highest confidence hypothesis
    • Confidence summary with reasoning explanation
    • Cleaner evidence display with optional next_check field

  • Enhanced Patch Spectrum: Risk-aware patch visualization improvements

    • [RECOMMENDED] marker for lowest-risk GO strategy
    • Files affected by each patch (extracted from diff)
    • Slop detection threshold adjusted (>50% instead of >60%)

Fixed

  • Critical: JSON Parser Enhancement - Copilot SDK timeout and response handling

    • Fixed missing timeout parameter in session.sendAndWait() (was using SDK default 60s, now explicit 90s)
    • Enhanced extractJsonObject() to handle markdown code blocks and various response formats
    • Improved error messages with response preview (first 300 chars) for debugging
    • Strengthened prompt to emphasize JSON format requirement
    • Parser now gracefully handles both pure JSON and JSON embedded in markdown

    Impact: Eliminates timeout errors and handles Copilot response format variations gracefully. Critical for production reliability.

    • Clear NO-GO explanation when all strategies flagged

  • Complete Workflow Summary: Guardian execution summary at completion

    • Total hypotheses generated
    • GO vs NO-GO strategy counts
    • All generated files listed with descriptions
    • Output directory prominently displayed

Changed

  • Progress messages now show what Guardian is doing, not just status
  • File creation logs include artifact names for audit trail clarity
  • Slop score threshold lowered to 50% for stricter quality control
  • Summary sections use bold headers for better visual hierarchy

Fixed

  • TypeScript type safety: ctx.headSha nullable handling
  • All progress messages now use consistent ASCII-safe formatting

Philosophy

This release embodies Guardian's core principle: "Trust built on receipts, not magic."

Every step of the AI's thinking process is now visible in terminal output, making it perfect for real execution recording (not demos). Judges and users can see exactly what Guardian is doing at every moment.

For Real Execution GIF Recording

npm run build
node dist/cli.js run --repo owner/repo --last-failed --show-reasoning --show-options

# Output now shows:
# [>] Checking GitHub MCP configuration...
# [+] GitHub MCP server ready
# [>] Fetching run context from GitHub...
# [+] Retrieved workflow logs and metadata
#     Workflow: .github/workflows/ci.yml
#     Commit SHA: a1b2c3d
# [>] Deep analysis: Extracting source context...
# [+] Found 2 source file(s) mentioned in errors
#     - src/utils.ts:45-55
#     - tests/integration.test.ts:120-125
# [>] Sending to Copilot for multi-hypothesis analysis...
#     (This may take 30-60 seconds)
# [+] Received response from Copilot
# ... (complete transparency continues)

[0.1.3] - 2026-02-09

🔧 Final Polish & SDK Terminology Alignment

Cosmetic fixes to align all user-facing messages with the SDK-based architecture.

Changed

  • Error Messages: Updated all "Copilot CLI" references to "Copilot SDK"
    • CopilotError now shows "Copilot SDK error:" prefix
    • Auth check shows "GitHub Copilot SDK: Available/Not available"
    • Install hint updated to npm install @github/copilot-sdk
  • README: Updated architecture diagram label "Copilot Chat API" → "Copilot SDK"
  • README: Renamed section "Five Layers of Copilot CLI Usage" → "Five Layers of Copilot SDK Usage"
  • CHANGELOG: Removed duplicate section header

[0.1.2] - 2026-02-09

🎯 SDK Integration Complete + Production Ready

This release represents the culmination of our SDK migration journey - a complete, battle-tested integration with comprehensive resource management and test coverage.

Highlights

  • Full SDK Lifecycle Management: All resource leaks eliminated
  • Test Coverage Solidified: 4 dedicated SDK tests with proper mocking
  • Production Robustness: Timer cleanup, promise reset, race condition handling

Fixed (Robustness - 4 LOW priority issues)

  • [LOW-1] Timeout Timer Leak: clearTimeout(timeoutId) now called on both success and error paths

    • File: async-exec.ts:229-255
    • Impact: No orphan timers in long-running sessions

  • [LOW-2] SDK Client Promise Reset: _sdkClientPromise = null on initialization failure

    • File: async-exec.ts:52-56
    • Impact: Retry capability after transient network failures

  • [LOW-3] closeSdkClient Race Condition: Await pending init before cleanup

    • File: async-exec.ts:63-76
    • Impact: Clean shutdown even during initialization

  • [LOW-4] Test Mock Isolation: resetMocks() in beforeEach

    • File: async-exec.test.ts:4,17
    • Impact: No test pollution between runs

Technical Notes

  • SDK session lifecycle: createSession() → send() → destroy() (always in finally block)
  • Timer management: Store timeoutId, clear in both try/catch paths
  • Promise state: Reset to null on rejection to enable retry
  • Shutdown sequence: await _sdkClientPromise → stop _sdkClient → reset promise

[0.1.1] - 2026-02-09

🚀 MAJOR: Copilot SDK Migration + Robustness Overhaul

This release marks a pivotal architectural shift and comprehensive hardening of the codebase.

The Journey: From CLI to SDK

Initially, we attempted to use gh copilot chat CLI subprocess spawning. However, extensive testing revealed this approach was fundamentally broken - the Copilot CLI extension does not support programmatic subprocess invocation the way we needed.

What we tried:

  • spawn('gh', ['copilot', 'chat', ...]) - No interactive mode support
  • Piped stdin/stdout - Response capture failed
  • Various timeout strategies - All ended in silent failures

The discovery: After researching official GitHub documentation and the Copilot CLI Challenge requirements, we discovered the @github/copilot-sdk - the proper way to integrate Copilot programmatically.

The migration:

  • Complete rewrite of async-exec.ts from subprocess spawning to SDK client
  • Session management with proper lifecycle (create → use → destroy)
  • Native promise-based async/await patterns

This journey demonstrates real-world engineering: recognizing when an approach is fundamentally flawed, researching alternatives, and executing a clean migration.

Added

  • @github/copilot-sdk Integration: Official SDK for Copilot API access

    • Singleton client pattern with lazy initialization
    • Per-request session management
    • Native timeout and retry handling

  • Resource Leak Prevention:

    • Timeout timer cleanup on both success and error paths
    • _sdkClientPromise reset on initialization failure (enables retry)
    • closeSdkClient() race condition handling (await pending init before cleanup)

  • Test Infrastructure for SDK:

    • __mocks__/@github/copilot-sdk.ts - Complete mock implementation
    • resetMocks() helper for test isolation
    • Dedicated SDK test cases (session destroy, empty response, timeout)

Fixed (SEVERE - 5 issues)

  • [S1] Global Install Support: Changed process.cwd() to PACKAGE_ROOT for prompt/schema loading

    • Files: analyze.ts, patch_options.ts, debug.ts (7 occurrences)
    • Impact: CLI now works when installed globally via npm install -g

  • [S2] qualityReview() Crash Prevention: Added try-catch for JSON parsing

    • File: patch_options.ts:145-159
    • Impact: Returns safe default instead of crashing on malformed Copilot responses

  • [S3] debugInteractive() Crash Prevention: Added try-catch with session recovery

    • File: debug.ts:66-79
    • Impact: User can retry instead of losing debug session

  • [S4] JSON Extraction Data Corruption: Replaced greedy regex with balanced brace parser

    • File: util.ts:63-104
    • Impact: No more silent JSON corruption from trailing text in Copilot responses

  • [S5] Over-Aggressive Secret Redaction: Removed 40+ char alphanumeric pattern

    • File: util.ts:40
    • Impact: Git SHAs, npm hashes, and diagnostic data preserved for analysis

Fixed (MODERATE - 2 issues)

  • [M1] MCP Token Configuration: Fixed literal string bug in all code paths

    • File: mcp.ts:110,124,139
    • Impact: MCP authentication now works for first-time users

  • [M3] File Path False Positives: Stricter regex with extension whitelist

    • File: context-enhancer.ts:21-31
    • Impact: Fewer wasted API calls, cleaner prompt context

Known Issues (Deferred)

  • [M2] interactiveApply() hardcoded choice: Deprecated function, low impact
  • [M4] confidence_score type mismatch: Best-effort mode handles gracefully
  • [L1-L3] Code cleanup: Optional improvements for future release

Technical Details

  • SDK Version: @github/copilot-sdk ^0.1.23
  • Model: gpt-4o (configurable via COPILOT_MODEL env var)
  • Audit Reference: PATCH_REPORT.md (SIDRCE SaaS v1.1.6)
  • Test Results: 41 passing, 18 skipped, 0 failures
  • Build: Clean TypeScript compilation
  • PACKAGE_ROOT: Uses __dirname for CommonJS compatibility

For Judges

npm install
npm run build
npm test
# ✅ Test Suites: 4 passed, 1 skipped, 5 total
# ✅ Tests: 41 passed, 18 skipped, 59 total
# ✅ Exit code: 0

Key Point: This project uses the official @github/copilot-sdk for Copilot integration, not CLI subprocess spawning. This is the correct approach per GitHub's official documentation.

[0.1.0] - 2026-02-03

✅ PRODUCTION READY - All Tests Passing

This release marks production readiness with comprehensive test improvements and enhanced error resilience.

Fixed

  • [CRITICAL] Null-safe error handling in copilotChatAsync - prevents crash on unexpected error types
  • [QUALITY] Test suite now passes cleanly: 38 passing, 18 documented skips, 0 failures
  • [CI/CD] Removed continue-on-error: true from GitHub Actions workflow
  • [RESILIENCE] Enhanced JSON parsing with graceful fallback and user-friendly error messages
  • [RESILIENCE] Added 3-layer defense against malformed LLM responses

Improved

  • Error Messages: All Copilot errors now include actionable hints (e.g., "Run: gh auth login")
  • Schema Validation: Best-effort fallback mode when non-critical fields are missing
  • Test Documentation: Added TEST_SUITE_UPDATE.md explaining test philosophy
  • Resilience Strategy: New docs/RESILIENCE_STRATEGY.md documenting error handling approach

Changed

  • Test Strategy: Migrated from brittle mocks to documented integration test skips
    • 38 unit tests verify core logic (100% passing)
    • 18 integration tests skipped with manual verification protocols
  • Quality Review: Improved mock reliability in patch_options tests
  • CI Signal: Tests now properly fail CI when broken (no silent failures)

Documentation

  • NEW: TEST_SUITE_UPDATE.md - Comprehensive test suite changes and justification
  • NEW: TESTING_PHILOSOPHY.md - Real-world first testing approach
  • NEW: docs/RESILIENCE_STRATEGY.md - Error handling and LLM failure mitigation
  • UPDATED: TEST_STATUS.md - Current test status with skip explanations

Technical Details

  • Test Coverage: 38/38 critical path tests passing
  • Exit Code: Clean exit (0) on npm test
  • Build: TypeScript compilation clean with strict mode
  • Integration: Manual verification protocols for all skipped tests

For Judges

npm install
npm run build
npm test
# ✅ Test Suites: 4 passed, 1 skipped, 5 total
# ✅ Tests: 38 passed, 18 skipped, 56 total
# ✅ Exit code: 0

[0.0.4] - 2026-02-02

Fixed

  • [CRITICAL] Replaced blocking execSync with async copilotChatAsync in debug.ts to prevent event loop blocking
  • [CRITICAL] Fixed debug transcript logging - now properly records Q&A pairs instead of empty templates
  • Enhanced MCP installation error messages with detailed troubleshooting guidance
  • Improved diff parsing to handle binary files, whitespace changes, and complex hunks
  • Better npm permission failure diagnostics for corporate/restricted environments

Improved

  • Debug interactive mode now fully asynchronous for better responsiveness
  • MCP setup provides clearer feedback for permission and PATH issues
  • Patch application more robust against edge cases (binary diffs, unusual formatting)

[0.0.3] - 2026-02-02

Fixed

  • Defensive Programming: Added null-safe handling for slop_score in patch output to prevent crashes
  • Test Alignment: Updated test mocks to match actual runtime behavior
    • Fixed copilotChatAsync command expectations in async-exec tests
    • Corrected fetchRunContext mock call order in github tests
    • Improved quality review mock completeness

Test Results

  • Test pass rate improved from 56% to 70% (39/56 passing)
  • Reduced failures from 24 to 16
  • Production code remains fully functional

[0.0.2] - 2026-02-02

Fixed

  • [CRITICAL] Fixed allowlist enforcement: patch_options now extracts affected files and passes them to applyPatchViaDiff
  • [CRITICAL] Enhanced diff parsing to detect deletions, renames, and modifications (not just additions)
  • [CRITICAL] Added deprecation warning to legacy autoHeal() text-replacement method
  • [SECURITY] Improved path safety validation using path.relative() for cross-platform consistency
  • [SECURITY] Enhanced MCP config merging to preserve existing non-mcpServers settings
  • [COMPATIBILITY] Replaced all Unicode checkmarks with ASCII equivalents for cp949 compatibility

Changed

  • applyPatchViaDiff now validates all diff operations (add/modify/delete/rename) against allowlist
  • Path safety checks now use path.relative() to prevent Windows case sensitivity issues
  • Legacy autoHeal() now emits deprecation warnings directing users to CLI --auto-heal mode
  • All console output converted to ASCII-safe characters ([+] instead of ✓)

Security

  • Closed path traversal vulnerability in diff application
  • Strengthened allowlist enforcement across all patching operations
  • Added comprehensive validation for delete and rename operations in diffs

[0.0.1] - 2026-02-02

Added

  • Core Analysis Engine: Multi-hypothesis reasoning system for CI/CD failure root cause analysis
  • Patch Generation: Three-strategy patch options (Conservative, Balanced, Aggressive) with risk assessment
  • Auto-Heal Mode: Automated patch application with retry logic and CI verification
  • MCP Integration: Model Context Protocol support for enhanced repository context
  • Anti-Slop Detection: Quality scoring system to detect and flag AI-generated bloat
  • Sovereign AI Philosophy: Full transparency with audit trails and user control
  • Beautiful CLI UI: Color-coded dashboard with confidence indicators and progress spinners
  • Comprehensive Testing: 43 tests covering async execution, analysis, patch generation, and auto-apply
  • Security Features: Secret redaction, path validation, and safe file operations
  • GitHub Actions CI/CD: Automated testing and build verification

Features

  • Fetch and analyze GitHub Actions failure logs via gh CLI
  • Generate structured analysis with hypothesis ranking and confidence scores
  • Create multiple patch strategies with quality verdicts
  • Interactive patch selection or automatic lowest-risk application
  • Real-time CI status monitoring with retry logic
  • Deep context injection using repository structure and source code
  • Debug mode for interactive troubleshooting
  • Persistent audit logs for all AI interactions

Documentation

  • Complete README with architecture diagrams (Mermaid)
  • API documentation and usage examples
  • Security policy and vulnerability reporting guidelines
  • Contributing guidelines for community collaboration
  • Before/After impact analysis
  • Visual storyboard and demo scenarios

Technical Details

  • Language: TypeScript 5.x
  • Runtime: Node.js 18+
  • Dependencies:
    • GitHub CLI (gh) for repository integration
    • GitHub Copilot CLI for AI-powered analysis
    • Chalk, Ora for terminal UI
    • Jest for testing
  • Architecture: Modular engine design with clear separation of concerns

Known Limitations

  • Requires GitHub CLI authentication (gh auth login)
  • Requires GitHub Copilot CLI installation
  • Auto-heal mode requires git repository context
  • MCP configuration may override existing Copilot CLI settings

Security

  • All logs are sanitized before AI processing
  • Path validation prevents directory traversal attacks
  • No credentials stored in project files
  • Audit trails maintained for compliance

Full Changelog: https://github.com/flamehaven01/copilot-guardian/commits/main