v0.1.8 — PII / PHI redaction

Opt-in PII / PHI redaction inside the LangChain audit handler. Sensitive values never reach storage; the SHA-256 chain signature is computed over the redacted payload, so an auditor reproducing the signature from a persisted record never needs access to raw secrets.

Added

• ragcompliance.redaction module with Redactor, Pattern, and 9 built-in patterns: email, ssn (with SSA never-issued prefix exclusion), credit_card (Luhn-validated), phone_us, ipv4, aws_access_key, openai_key, anthropic_key, bearer_token. Custom patterns supported.
• 3 new config fields and env vars: RAGCOMPLIANCE_REDACT_PII, RAGCOMPLIANCE_REDACTION_PATTERNS, RAGCOMPLIANCE_REDACTION_REPLACEMENT.
• Per-record extra["redaction_findings"] surfaces per-pattern hit counts to dashboards and SOC 2 reports without persisting raw values.
• 27 new tests. 0 regressions on the existing suite. 171 total passing.

Changed

• RAGComplianceHandler.on_chain_end runs the redactor before _sign_chain. Hot path is bit-for-bit identical when redact_pii=False (the default), so upgrading from 0.1.7 is a no-op.

Install

pip install --upgrade ragcompliance==0.1.8

v0.1.7

This release is a credibility and polish pass. No handler behaviour changes. The library surface, signatures, and audit semantics are identical to 0.1.6; everything that moved is either packaging, docs, or operator tooling around the install.

Added

• SECURITY.md at the repo root documenting supported versions, private disclosure channel (daksh.trehan@hotmail.com), response SLAs, scope, and safe-harbour terms. Required for responsible disclosure and for landing a GitHub Security Advisory.
• ragcompliance-selftest console script (new module ragcompliance.selftest) that verifies an install is wired correctly: import + version readback, optional extras probe, required env vars, RAGComplianceConfig.from_env() load, and a dev-mode storage round-trip confirming SHA-256 signature recomputation. Red / yellow / green severity, --json for CI pipelines, --dev-mode to force the stdout path on a fresh machine.
• ragcompliance-soc2 console script wired to the existing ragcompliance.soc2._main entry point. Operators no longer need to know the module path to generate an evidence report.
• Docs site FAQ section (six questions: LangSmith delta, PHI / PII, legal signature, encryption, Supabase-only, Haystack / DSPy).
• Comparison strip on the landing page (Works with / Complements / Storage). Disambiguates the stack position before the problem statement so LangSmith users stop treating this as a competitor.
• Billing reference-implementation callout in the docs. The Stripe plans shipped in ragcompliance.billing are a fork-to-your-own-billing reference, not a paid tier of this library.
• Changelog URL in [project.urls] and a footer link on the landing page.

Changed

• Em-dash sweep across README.md, docs/index.html, and docs/docs.html. All em-dashes and en-dashes replaced with semicolons, parentheses, colons, or sentence splits. Zero dashes remain across the three surfaces.
• Landing page now uses dynamic shields.io badges for PyPI version and GitHub Actions CI status instead of hardcoded strings. Every future release updates the badges automatically.
• Landing page test-count references updated from 126 / 145 to 152 (current suite). Pilot-customer language removed; testimonial quotes reframed as illustrative customer-question archetypes (the audit-reconstruction ask, the evidence-shape ask, the answer-drift ask).
• Softened the "40 to 60 percent of RAG projects never reach production" stat to "A lot of RAG projects stall before production". The original number is widely cited but not defensible in an auditor conversation.
• Dashboard mockup on the landing page now carries an example / mock data label and an aria-label so no reader mistakes the screenshot for live production data.
• Example model_name in README.md unified to gpt-4o-mini everywhere (was a mix of gpt-4 and gpt-4o-mini).

Internal

• None.

v0.1.6 — Clarifications: latency scope + SOC 2 sample-size default

v0.1.6 — Clarifications: latency scope + SOC 2 sample-size default

Small patch release. No code-logic changes, all cosmetic and docs except for the SOC 2 sample-size default bump.

Changed

• SOC 2 evidence report default sample size raised from 5 to 25. The previous default of 5 was symbolic. On a workspace with 50k records it gave roughly a 1-in-10,000 shot of surfacing a rare tamper event — fine for a demo, wrong for a quarterly spot-check that an auditor is going to look at. 25 is still fast on realistic volumes (~3s on 2000 records in benchmarks) but lands in statistically meaningful territory. The CLI default (--sample) and the Python API default (generate_report(..., sample_size=25)) both updated.
• README and docs site now distinguish "handler overhead in isolation" (<1ms, ~38µs p50) from "end-to-end chain latency" (dominated by retriever and LLM). No behavior change; the prior claim was correct but ambiguous enough that a reader could misread the full-chain number as the handler's contribution.

Added

• README "Sample size and confidence" subsection explaining the hypergeometric tradeoff between sample size and detection probability for SOC 2 evidence audits, plus a programmatic note on exhaustive verification for deeper due-diligence runs.
• 2 new tests in tests/test_soc2.py::TestDefaults that assert the generate_report default and the CLI --sample default stay at 25. Regression guards so a future drive-by change does not silently revert the default and surface stale evidence to auditors.

Full changelog: https://github.com/dakshtrehan/ragcompliance/blob/main/CHANGELOG.md
Diff: v0.1.5...v0.1.6

v0.1.5 — Retriever-chunk capture on langchain-core >=1.3.0

v0.1.5 — Retriever-chunk capture on langchain-core >=1.3.0

Fixed

• Retrieved chunks silently dropped on langchain-core >= 1.3.0. The handler only overrode on_chain_start, so on_retriever_start fired with a parent_run_id that was never recorded in _run_parents. When the matching on_retriever_end fired, _resolve_root could not walk up to the tracked root audit state and every retrieved chunk was silently dropped from the record. The record still saved with a valid SHA-256 signature but retrieved_chunks was an empty list — a silent correctness bug for any RAG chain using the recommended LCEL pattern. Earlier LangChain versions happened to double-fire on_chain_start for retrievers so on_chain_start's parent registration covered them incidentally; langchain-core 1.3.0 tightened the callback surface and removed that side channel. on_retriever_start, on_llm_start, on_chat_model_start, and on_tool_start now all register their run_id/parent_run_id pairs through a shared _register_descendant helper so every *_end event can route back to the correct audit state.

Added

• 5 new tests in tests/test_handler_retrieval.py: LCEL baseline, batch with per-invocation chunks, deep-nested retriever, on_llm_start parent-registration guard, and 10-thread concurrent invoke with chunks. All five fail against the v0.1.4 handler (verified by monkey-patching the new overrides back to BaseCallbackHandler's no-ops), so they are real regression guards.

Changed

• on_chain_start now delegates inner-runnable parent tracking to the new _register_descendant helper. Pure refactor — same behavior as v0.1.4 for the chain-start path.

Full changelog: https://github.com/dakshtrehan/ragcompliance/blob/main/CHANGELOG.md
Diff: v0.1.4...v0.1.5

v0.1.4 — batch() correctness, thread-safe handler, defensive storage

The correctness release. If you have been running chain.batch([...]) or sharing a handler across concurrent chain.invoke calls, upgrade immediately.

See CHANGELOG.md for the full entry.

The bug this fixes

Prior to v0.1.4, both handlers kept per-invocation state on the instance. With chain.batch([q1, q2, q3]) or concurrent invokes on a shared handler, only the outermost chain's record would save, and fields bled across runs — the first query could be paired with the last query's answer. For a compliance library, silent audit loss with fabricated pairings is the most dangerous possible failure mode.

What shipped

• Per-run state in the LangChain handler. State now lives in self._runs[run_id] = _RunState, guarded by a threading.Lock, with parent_run_id resolving to the tracked root via self._run_parents. Each _RunState tracks its descendants for O(1) cleanup on chain end.
• Per-trace state in the LlamaIndex handler. State keyed by trace_id in self._traces, with active trace routed via threading.local so the trace_id-less on_event_* callbacks reach the correct state.
• Defensive storage.save(). Both handlers wrap self.storage.save(record) in try/except. A misbehaving custom storage can no longer kill a chain or leak pending state.
• New env var RAGCOMPLIANCE_MAX_PENDING_RUNS (default 10000): soft cap on in-flight root run states with oldest-first eviction. Guards against memory leaks if on_chain_end is never delivered.

Test suite: 145 passed (up from 126)

19 new regression tests covering:

• 3-query batch with correctly-paired records
• 10 and 20 concurrent threads on a shared handler
• Nested LCEL chains saving exactly once
• Inner events routing to root state
• Interleaved event ordering
• Soft-cap eviction with descendant cleanup
• Raising storage not killing the chain
• on_chain_end p50 hot-path microbench
• Bad env var values falling back to default
• LlamaIndex: 10 concurrent traces producing distinct records

Install

pip install --upgrade ragcompliance==0.1.4

Upgrading from 0.1.3

No breaking API changes for normal users. If you were reaching into handler internals (e.g. handler._query), those attributes no longer exist — state now lives in handler._runs[run_id]. The public callback API is unchanged.

v0.1.3 — Stripe live-mode readiness + OSS launch

The OSS-launch release. RAGCompliance is MIT-licensed middleware for putting RAG chains on an audit trail. See CHANGELOG.md for the full entry.

Highlights

• Stripe live-mode readiness probe (/health/billing) so you find out your keys are not wired before a Saturday-night outage, not during one.
• Landing page + full docs at www.dakshtrehan.com/ragcompliance.
• Repositioned as pure OSS with optional paid support — no paid tier inside the project itself.
• Dashboard detail endpoint now does an indexed lookup instead of an in-memory scan. Records older than the previous 500-record window are now reachable; the endpoint still returns 404 on unknown ids.
• Signature coverage spelled out in the README and the SOC 2 CC8.1 claim so you know exactly what fields are inside the SHA-256 and which are out of scope.

Install

pip install ragcompliance==0.1.3

Upgrading from 0.1.2

No breaking API changes. /api/logs/detail/{id} now returns 404 for unknown ids (previously 404 for ids outside the 500-record window). If you were relying on that second behavior, do not.

Note: skip ahead to v0.1.4 if you are using chain.batch() or sharing a handler across concurrent invokes.

v0.1.2 — SOC 2 evidence, SSO, Slack alerts, async writes

The compliance-layer release. See CHANGELOG.md for the full entry.

Added

• SOC 2 evidence report generator (ragcompliance.soc2) that pulls live audit records, recomputes signatures on a random sample, and renders a Markdown report mapped to CC6.1, CC7.2, CC8.1, A1.1, C1.1 with a methodology section. Not an attestation, but the evidence pack an auditor asks for on day one.
• Opt-in OIDC SSO on the dashboard via the sso extra (pip install ragcompliance[sso]). /health and /stripe/webhook stay open by design; everything else returns 401 when SSO is configured and the session is anonymous.
• Slack alerts on four anomaly rules: retrieval_returned_zero_chunks, low_similarity, chain_slow, chain_errored. Alerts are fire-and-forget so an alerting outage cannot suppress the audit record itself.
• Async audit writes with a bounded queue, atexit-based shutdown drain, and a flush() method for tests and explicit app-shutdown hooks. The chain hot path no longer blocks on Supabase.

Install

pip install ragcompliance==0.1.2

Note: skip ahead to v0.1.4 if you are using chain.batch() or sharing a handler across concurrent invokes.

v0.1.1 — LCEL safety, quota blocking, Stripe webhook dict normalization

The stability release. See CHANGELOG.md for the full entry.

Fixed

• LCEL safety. LCEL pipelines fire on_chain_start / on_chain_end for every sub-runnable. The handler now latches the outermost chain via run_id / parent_run_id so state from nested runnables no longer overwrites mid-chain and produces half-built audit records.
• Quota enforcement now actually blocks the chain when a workspace is over its plan quota and enforce_quota=True. The RuntimeError raised in on_chain_start propagates out (raise_error = True).
• Stripe webhook handler normalizes incoming event objects to plain dicts before indexing, fixing spurious TypeError on live-mode events from Stripe CLI.
• Billing period reset wired into the quota check so usage counters actually roll over when Stripe's current_period_end passes.
• LlamaIndex handler payload keys aligned with the LangChain handler so the SHA-256 signature is identical across both integrations.

Install

pip install ragcompliance==0.1.1

Note: skip ahead to v0.1.4 if you are using chain.batch() or sharing a handler across concurrent invokes.

v0.1.0 — Initial release

The initial release of RAGCompliance. See CHANGELOG.md for the full entry.

What shipped

• LangChain callback handler (RAGComplianceHandler) that captures query, retrieved chunks with source URLs and similarity scores, LLM answer, model name, and latency, then signs everything with SHA-256 and writes one row per invocation to the rag_audit_logs table.
• LlamaIndex callback handler (ragcompliance.llamaindex_handler.LlamaIndexRAGComplianceHandler) mirroring the LangChain capture shape via the optional ragcompliance[llamaindex] extra.
• AuditStorage backed by Supabase with row-level security per workspace, dev-mode fallback that prints records to stdout, and query() for paged reads.
• FastAPI dashboard (ragcompliance.app) with stats cards, recent logs, filterable CSV / JSON export, detail endpoint, and a liveness probe. Installed via ragcompliance[dashboard].
• Stripe billing and quota metering: tier-aware quotas, checkout session creation, webhook-driven subscription state, and a self-hostable paid-tier UI as a reference implementation.
• RAGComplianceConfig with from_env() and sensible defaults so the middleware works out of the box in dev.

Install

pip install ragcompliance==0.1.0

Note: if you are starting today, skip ahead to v0.1.4 — it fixes a critical chain.batch() correctness bug that affected every prior version.