Add guardrails and update audit sources for cargo-vet workflow#803
Add guardrails and update audit sources for cargo-vet workflow#803jerrysxie wants to merge 3 commits intoOpenDevicePartnership:mainfrom
Conversation
Add duplicate-check guidance to the cargo-vet skill workflow. Add pre-certification guardrails to the cargo-vet auditor agent. This helps prevent repeated certify retries from appending identical audits.
Add non-interactive execution rules (PAGER=cat, --accept-all) to the agent to prevent commands from blocking on terminal input. Add exemption-as-last-resort policy to both the skill and agent, requiring explicit user confirmation and notes for every exemption.
Remove the embark-studios and zcash third-party audit import sources which are no longer trusted. Add first-party audits for the crates previously covered by those imports: - autocfg 1.4.0 -> 1.5.0 (safe-to-deploy, delta) - crunchy 0.2.3 -> 0.2.4 (safe-to-deploy, delta) - serde_spanned 0.6.8 -> 0.6.9 (safe-to-deploy, delta) - tap 1.0.1 (safe-to-deploy, full) - thread_local 1.1.4 -> 1.1.9 (safe-to-deploy, delta) - valuable 0.1.1 (safe-to-deploy, full) - rustc-demangle (trusted author entry)
jerrysxie
requested review from
RobertZ2011,
felipebalbi,
makubacki,
tullom and
williampMSFT
There was a problem hiding this comment.
Pull request overview
This PR tightens the repository’s cargo-vet workflow by removing unused third-party import sources, adding/refreshing first-party audit coverage for crates that previously relied on those imports, and documenting new reviewer guardrails (non-interactive execution, exemption policy, and duplicate-audit deduping).
Changes:
- Removed unused cargo-vet import sources (EmbarkStudios, Zcash) and refreshed
imports.lockaccordingly. - Added first-party audit records for several crates and introduced a new trusted publisher entry for
rustc-demangle. - Updated internal cargo-vet reviewer docs/skills with stricter rules for exemptions and duplicate-audit prevention, plus non-interactive command guidance.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
supply-chain/config.toml |
Removes unused import sources to reduce reliance on external audit registries. |
supply-chain/imports.lock |
Refreshes imported audit/publisher data to match updated import sources and coverage strategy. |
supply-chain/audits.toml |
Adds new first-party audits and a new trusted publisher entry to maintain/extend coverage after import removal. |
.github/skills/cargo-vet-audit/SKILL.md |
Documents stricter exemption policy and introduces a duplicate-audit deduping step. |
.github/agents/cargo-vet-auditor.agent.md |
Adds guardrails for non-interactive cargo-vet usage, exemption policy, and duplicate-audit checks. |
| ## Exemptions Are a Last Resort | ||
|
|
||
| Do **not** recommend adding `[[exemptions]]` entries without good reason. | ||
| Each exemption bypasses the audit process entirely and requires explicit manual | ||
| confirmation from the user. Always prefer performing a full or delta audit over | ||
| exempting a crate. If an exemption is truly necessary (e.g., the crate is only | ||
| needed at `safe-to-run` level for dev tooling, or an import source removal | ||
| requires temporary coverage), flag it clearly and let the user decide. | ||
|
|
||
| Every exemption **must** include a `notes` field explaining why the exemption | ||
| exists and under what conditions it can be removed. | ||
|
|
||
| ## Duplicate-Audit Guardrail | ||
|
|
||
| Before recommending or running certification, check whether an identical | ||
| `[[audits.<crate>]]` entry (same who/criteria/version-or-delta/notes) already | ||
| exists in `supply-chain/audits.toml`. | ||
|
|
||
| If an identical entry already exists: | ||
|
|
||
| - Do not recommend re-certifying with the same data | ||
| - Report that the crate is already certified with identical audit content | ||
| - If duplicates already exist, explicitly recommend deduplicating by keeping one | ||
| copy and removing the rest | ||
|
|
There was a problem hiding this comment.
The new sections “Exemptions Are a Last Resort” and “Duplicate-Audit Guardrail” are introduced as ## headings, which makes them siblings of “## How to Review” rather than subsections of it. This breaks the document hierarchy/TOC; these should be ### (or otherwise nested) under “How to Review”, consistent with the surrounding structure.
| - **`diff` / `inspect`:** Always set the pager to `cat` to prevent the pager | ||
| from waiting for input. Use `$env:PAGER='cat';` (PowerShell) or `PAGER=cat` | ||
| (POSIX) before the command. |
There was a problem hiding this comment.
The PowerShell guidance $env:PAGER='cat' may not actually prevent blocking on Windows because cat is typically a PowerShell alias, not an external executable that a spawned process (cargo-vet) can invoke via PAGER. Consider updating the Windows/PowerShell recommendation to a pager that is reliably available as an executable (e.g., more.com), or explicitly note that cat requires a Unix-like toolchain in PATH (Git Bash/MSYS).
| - **`diff` / `inspect`:** Always set the pager to `cat` to prevent the pager | |
| from waiting for input. Use `$env:PAGER='cat';` (PowerShell) or `PAGER=cat` | |
| (POSIX) before the command. | |
| - **`diff` / `inspect`:** Always set the pager to a non-interactive command to | |
| prevent the pager from waiting for input. Use `$env:PAGER='more.com';` | |
| (PowerShell) or `PAGER=cat` (POSIX) before the command. |
| Suggested duplicate detection commands: | ||
|
|
||
| ```powershell | ||
| # PowerShell: use any local script/command that prints duplicate blocks | ||
| # with crate names and line numbers | ||
| ``` | ||
|
|
||
| ```shell | ||
| # POSIX: optional equivalent using awk/python if available | ||
| # (implementation may vary by environment) | ||
| ``` |
There was a problem hiding this comment.
This section introduces “Suggested duplicate detection commands” but the PowerShell/POSIX code blocks are placeholders and don’t include any usable command. Either provide a concrete minimal example (even if optional) or reword/remove the “Suggested … commands” wording so readers aren’t left without actionable guidance.
This pull request introduces several important improvements to the cargo-vet audit process documentation and configuration, focusing on stricter exemption policies, duplicate audit entry prevention, and the addition of new audit records. The main themes are: enhanced documentation for reviewers, stricter exemption and deduplication workflows, and updates to the audit registry.
Documentation and Workflow Improvements:
cargo vetcommands non-interactively to prevent blocking on terminal input, and clarified how to set the pager and use--accept-allfor certification. (.github/agents/cargo-vet-auditor.agent.md)[[exemptions]]) should only be used as a last resort, requiring explicit user confirmation and justification, and that every exemption must include anotesfield explaining its necessity and removal conditions. (.github/agents/cargo-vet-auditor.agent.md, .github/skills/cargo-vet-audit/SKILL.md) [1] [2]Audit Registry Updates:
autocfg,crunchy,serde_spanned,tap,thread_local, andvaluable, all with detailed notes and criteria. (supply-chain/audits.toml) [1] [2] [3] [4] [5]rustc-demanglewithsafe-to-deploycriteria. (supply-chain/audits.toml)Configuration Clean-up:
supply-chain/config.toml(specificallyembark-studiosandzcash). (supply-chain/config.toml)