Skip to content

Commit 878deb0

Browse files
committed
feat: add OSV IOC candidate pipeline scaffold
1 parent de7a4bf commit 878deb0

7 files changed

Lines changed: 817 additions & 0 deletions

File tree

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
name: IOC Candidate Monitor
2+
3+
on:
4+
schedule:
5+
- cron: "15 10 * * *"
6+
workflow_dispatch:
7+
8+
permissions:
9+
contents: write
10+
pull-requests: write
11+
12+
jobs:
13+
generate-candidates:
14+
name: Generate OSV blacklist candidates
15+
runs-on: ubuntu-latest
16+
17+
steps:
18+
- uses: actions/checkout@v4
19+
20+
- name: Set up Go
21+
uses: actions/setup-go@v5
22+
with:
23+
go-version-file: go.mod
24+
25+
- name: Fetch IOC candidates
26+
run: |
27+
go run ./scripts/ioc-candidates \
28+
-since 24h \
29+
-min-severity HIGH \
30+
-ecosystems npm,PyPI,Go \
31+
-out /tmp/candidates.json \
32+
-existing pkg/analyzer/data/blacklist.json
33+
34+
- name: Count candidates
35+
id: candidate_count
36+
run: |
37+
count="$(jq 'length' /tmp/candidates.json)"
38+
echo "count=$count" >> "$GITHUB_OUTPUT"
39+
40+
- name: Merge candidates into blacklist
41+
if: steps.candidate_count.outputs.count != '0'
42+
run: |
43+
jq -s '
44+
(.[0] + .[1])
45+
| unique_by("\(.ecosystem)|\(.component)|\(.affected_versions | join(","))")
46+
| sort_by(.ecosystem, .component)
47+
' pkg/analyzer/data/blacklist.json /tmp/candidates.json > /tmp/merged-blacklist.json
48+
mv /tmp/merged-blacklist.json pkg/analyzer/data/blacklist.json
49+
50+
- name: Create pull request
51+
if: steps.candidate_count.outputs.count != '0'
52+
uses: peter-evans/create-pull-request@v7
53+
with:
54+
token: ${{ secrets.TOOLTRUST_BOT_TOKEN || github.token }}
55+
branch: ioc-candidates/${{ github.run_id }}
56+
title: "ioc: ${{ steps.candidate_count.outputs.count }} new blacklist candidate(s)"
57+
commit-message: "data(blacklist): auto-append ${{ steps.candidate_count.outputs.count }} OSV candidate(s)"
58+
body: |
59+
Automated IOC blacklist candidates generated from OSV ecosystem feeds for the last 24 hours.
60+
61+
Review each entry carefully:
62+
- Is the version pinning exact and narrow enough?
63+
- Is `BLOCK` the right action, or should this be downgraded to `WARN`?
64+
- Is the reason clear enough for someone triaging a finding?
65+
66+
Close this PR if any candidate looks incorrect. The workflow will retry on the next scheduled run.
67+
labels: |
68+
ioc
69+
automated

docs/ioc-pipeline.md

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# IOC Blacklist Auto-Candidate Pipeline
2+
3+
This workflow adds a daily review loop for likely blacklist entries without silently editing the live scanner data.
4+
5+
## What it does
6+
7+
- pulls recent OSV ecosystem advisories for `npm`, `PyPI`, and `Go`
8+
- filters to advisories published in the last 24 hours
9+
- keeps only `HIGH` and `CRITICAL` candidates
10+
- skips package versions already present in [`pkg/analyzer/data/blacklist.json`](/Users/brian93512/projects/tooltrust-scanner/pkg/analyzer/data/blacklist.json)
11+
- opens one review PR with candidate blacklist entries
12+
13+
The workflow never auto-merges. A human still decides whether a candidate belongs in the enforced blacklist.
14+
15+
## Why this exists
16+
17+
Our blacklist is strong once an entry exists, but hand-editing it means we can lag major supply-chain events by hours. This pipeline narrows that gap by surfacing review-ready candidates quickly after OSV publishes an advisory.
18+
19+
## Review checklist
20+
21+
When the workflow opens a PR:
22+
23+
1. confirm the affected versions are exact and narrow enough
24+
2. confirm `BLOCK` is the correct action
25+
3. rewrite the reason if the current summary is too vague for triage
26+
4. close the PR if the advisory is too broad, too noisy, or otherwise not suitable for blacklist enforcement
27+
28+
## Local dry run
29+
30+
```bash
31+
go run ./scripts/ioc-candidates \
32+
-since 720h \
33+
-min-severity HIGH \
34+
-ecosystems npm,PyPI,Go \
35+
-out /tmp/candidates.json \
36+
-existing pkg/analyzer/data/blacklist.json
37+
```
38+
39+
That gives us a 30-day sample so we can inspect candidate quality before relying on the scheduled workflow.
40+
41+
## Failure behavior
42+
43+
- transient OSV fetch failures log a warning and produce an empty candidate set
44+
- the workflow does not fail `main` because an upstream feed had a bad day
45+
- a no-op day simply means no PR is opened
46+
47+
## Scope
48+
49+
This is intentionally narrow:
50+
51+
- it only proposes additions to the blacklist
52+
- it does not auto-remove entries
53+
- it does not cover pre-advisory blog posts or social-media disclosures
54+
- it does not replace the existing threat-intel issue workflow
55+
56+
Those are follow-ups once candidate quality is stable.

0 commit comments

Comments
 (0)