ARES-AGENT

Autonomous Solana program security auditor built on LangGraph (JS), OpenRouter, a five-level "Crystalline" cognitive memory layer, and a hybrid (Supabase + Neo4j) knowledge base.

ARES runs an audit as a graph of phases. The analysis phase fans out to several analyzers in parallel, and every audit both reads from and writes back to a growing body of security knowledge.

Architecture

        intake ──► recall ──┬──► analyzeOnchain   ──┐
        (LLM)      (hybrid)  ├──► analyzeStatic    ──┤
                            ├──► analyzeHeuristic  ──┼──► merge ──► remember ──► report
                            └──► analyzeCua (opt-in)─┘   (rank)     (persist)    (LLM)

Phase	What it does
INTAKE	LLM parses the request into a structured target/depth/concerns summary.
LOAD-SOURCE	Reads the target's .rs and Anchor IDL files into context, bounded by ARES_SOURCE_BUDGET_CHARS. Shared by every analyzer.
RECALL	Hybrid retriever pulls relevant prior knowledge (see below).
ANALYZE	Four analyzers run in parallel and append findings:
	· onchain — reads the program account's deployment metadata via Solana/Helius RPC (owner, loader, size, upgrade authority). It does not disassemble the deployed bytecode, so its findings are about deployment posture, not program logic.
	· static — runs Semgrep over the source path using the committed ruleset in rules/ (optional binary; a scan that fails is reported as failed, never as clean).
	· heuristic — LLM reasoning over the loaded program source, plus intake and recalled memory. Findings must cite a file that was actually read; ones that don't are demoted to speculative. With no readable source the analyzer runs black-box and all its findings are demoted.
	· cua — opt-in: drives a real browser (Scrapybara Computer Use Agent)
	to investigate explorers/repos/docs. See below.
MERGE	Fan-in join: dedupes and severity-ranks the combined findings.
VERIFY	Skeptical critic pass: refines confidence/status, drops false-positives.
REMEMBER	LLM decides what to persist; writes crystals + runs consolidation.
REPORT	Synthesizes a professional audit report (severity matrix, stable finding IDs, coverage, analyzer status).

Analyzer status & incomplete assessments

Every analyzer degrades gracefully — an RPC error, a missing Semgrep binary, or unparseable model output all end with zero findings. That makes a broken run look exactly like a clean one, so each analyzer reports an outcome on the analyzers state channel:

Outcome	Meaning
ok	Ran against real input. Silence here is evidence.
skipped	Not applicable (no target of its kind, opt-in off).
degraded	Ran with a missing capability or unusable output.
failed	Attempted and errored. Silence here means nothing.

REPORT renders these as a table in Scope & Methodology, and when any analyzer is degraded or failed it prepends a warning banner to the finished report. The banner is added in code after synthesis (src/graph/analyzer-status.ts), so it cannot be dropped by the model.

Hybrid retrieval (RECALL)

Recall unions three sources and merges their scores, so a fragment surfaced by several sources ranks higher:

1. Crystalline — in-process activation-based memory (working/episodic recall). Recalled fragments are activated (activation boosted, access count incremented), which is what makes the episodic→semantic promotion in consolidate() reachable. Fragments synthesized from Supabase/Neo4j are skipped — they have no crystal in the store.
2. Supabase — hybrid_search RPC over pgvector + full-text (RRF) for candidate retrieval.
3. Neo4j — a standalone lexical match over graph chunks plus 1–2 hop graph expansion / relationship-aware reranking of the Supabase candidates.

Every source degrades gracefully: with Supabase/Neo4j/embeddings unset, recall falls back to Crystalline-only and the agent still runs fully offline.

Because a source that errors returns no fragments — exactly like one that had nothing to offer — each source's outcome is recorded on the retrieval state channel and rendered in the report. An unconfigured source is skipped (the documented default); a configured one that failed is failed, and that adds a line to the assurance banner saying prior audit knowledge was unavailable.

CUA investigation analyzer (opt-in)

analyzeCua drives a real, Scrapybara-hosted browser to gather external evidence about the audit target (block explorers, source repos, docs, prior audit mentions), then turns the investigation transcript into findings.

• Opt-in and off by default. Enable with CUA_ENABLED=true or pass --cua for a single run. It only activates when both OPENAI_API_KEY and SCRAPYBARA_API_KEY are set — otherwise the node returns no findings immediately, exactly like analyzeStatic/analyzeOnchain do without their inputs, so the graph and test suite stay hermetic by default.
• Uses OpenAI directly, not OpenRouter. @langchain/langgraph-cua invokes OpenAI's computer-use-preview model itself (via ChatOpenAI, which reads OPENAI_API_KEY from the environment) — there's no way to route the browser-driving loop through OpenRouter. The rest of ARES is unaffected and stays on OpenRouter.
• Read-only by design. The system prompt (cuaInvestigationSystemPrompt in src/llm/prompts.ts) explicitly forbids authentication, form submission, and any other state-changing action — the agent may only navigate and read.

Persistence

• Checkpointer: PostgresSaver persists per-thread graph state across runs.
• Crystalline store: InMemoryStore (session-scoped) — LangGraph JS has no Postgres store yet, so durable cross-audit knowledge lives in the Supabase + Neo4j knowledge base instead.
• Knowledge writeback: REMEMBER writes each persisted fragment to Crystalline and, when Supabase/Neo4j are configured, durably to the hybrid KB (src/persistence/knowledge-writer.ts) using the same doc_id/chunk_id scheme as the seed ingester — so runtime-learned knowledge survives restarts and is recalled in later audits. Without those backends it stays Crystalline-only, exactly as before.

Setup

cp .env.example .env      # fill in OPENROUTER_API_KEY (rest are optional)
npm install
npm run db:up             # starts Postgres + Neo4j via docker compose
npm run db:migrate        # creates checkpoint tables + Neo4j constraints

Supabase schema is applied separately (cloud or supabase CLI):

# apply db/supabase/0001_hybrid_search.sql via the Supabase SQL editor / CLI

Seed the knowledge base from the solsec corpus (requires Supabase and/or Neo4j credentials):

npm run ingest:solsec

Run an audit

# On-chain program (uses Helius RPC when HELIUS_RPC_URL is set):
npm run audit -- --program <PROGRAM_ADDRESS>

# Local source with Semgrep static analysis:
npm run audit -- --source ./path/to/program

# Quick local run without Postgres:
npm run audit -- --program <ADDRESS> --ephemeral

# Opt into the CUA browser-investigation analyzer for this run
# (requires OPENAI_API_KEY + SCRAPYBARA_API_KEY):
npm run audit -- --program <ADDRESS> --cua

Development

npm run typecheck   # tsc --noEmit
npm run lint        # eslint (flat config)
npm test            # vitest — hermetic; no external services required
npm run build       # emit to dist/

The test suite runs the full graph end-to-end with a fake LLM and an in-memory store, exercising the parallel fan-out, the concat-reducer findings channel, and Crystalline persistence — with Supabase/Neo4j unset to prove graceful fallback.

Continuous integration

.github/workflows/ci.yml runs the four commands above (typecheck, lint, build, test) on every push and pull request to main, across Node 20 and 22. Because the suite is hermetic, CI needs no secrets or services. Dependency updates are grouped into weekly Dependabot PRs (.github/dependabot.yml).

Two further jobs cover the evaluation harness: eval-scorer runs its pytest suite, and verify-claims fetches the ground truth and checks that the release gate still accepts and rejects at the right F1. See Detection accuracy.

Detection accuracy

ARES's detection accuracy is unmeasured. No prediction output has been scored against a labeled dataset, so no precision, recall, or F1 figure for this system is published — including the 0.94 F1 that has been quoted internally.

Metric	Value	Status
Precision	not measured	unverified
Recall	not measured	unverified
F1	not measured	unverified

The harness to measure it is in eval/: fetch_datasets.py builds a 152-label ground truth set from FraChiacc99/solana-vuln-rust, score_detections.py scores audit output against it, and the verify-claims CI job fails a release event while eval/predictions/ares-latest.csv is absent. The table above is updated from that job's output, not by hand.

What has been measured: static-ruleset output on audited production code

This is not an accuracy figure and does not belong in the table above. Recall was not measured at all — nobody established which real vulnerabilities are present in these programs — and precision was not computed either: no finding here has been adjudicated true or false positive. What it measures is how much output rules/solana.yml produces when pointed at Solana programs that have already passed professional audit, where most output is therefore expected to be false. This is the static layer alone — the LLM heuristic analyzer, the on-chain analyzer and VERIFY were not run — and the commits in the table below are each repo's HEAD on the measurement date, not the commits the published audits covered, so some of the scanned code post-dates its audit.

Over 917 files from 10 audited programs, the committed ruleset emits 22 findings (24.0 per 1k files): 21 non-canonical-bump, 1 account-close-revival.

Program	Commit	Files	Findings
drift-labs/protocol-v2	13e8e9b	187	0
blockworks-foundation/mango-v4	ee671d2	200	0
marinade-finance/liquid-staking-program	b8fe3f8	53	6
Ellipsis-Labs/phoenix-v1	5a34f7f	45	1
Squads-Protocol/v4	c34015c	41	4
mrgnlabs/marginfi-v2	2cc4e06	158	1
metaplex-foundation/mpl-token-metadata	349e061	114	0
solendprotocol/solana-program-library	d04ce00	24	10
solana-program/token	cd5cdc8	7	0
wormhole-foundation/wormhole	49f4295	88	0

Reproduce a row with the same invocation src/tools/semgrep.ts issues, pointed at that repo's program subtree — the first of programs/, program/, solana/programs/, solana/ that contains .rs files, else the checkout root. Files is the count of .rs files semgrep opened under that subtree, and it is the denominator of every per-1k rate here:

semgrep --json --quiet --metrics=off --no-git-ignore \
  --config rules/solana.yml <program-checkout>/<subtree>

Measurement is why the ruleset shrank from 7 rules to 4. Run over 805 files from 8 of these programs — before the solend and wormhole clones finished — the then-6-rule set emitted 386 findings, 479.5 per 1k files, on code that had already been audited. The two runs cover different corpora, so the drop from 479.5 to 24.0 per 1k is not a like-for-like before/after on fixed code: solend alone supplies 10 of the 22 findings that remain, and it was not in the earlier corpus. Read it as the order-of-magnitude effect of dropping two rules. Three rules have been removed in total, each recorded with its reasoning in rules/solana.yml:

• anchor-constraint-gap produced 370 of those 386 (96% of all output). Classified by regex over the four lines above each hit, not by manual review: 50.0% already carried the #[account(...)] constraint the rule's own message told them to add, 31.4% carried the /// CHECK: justification Anchor refuses to compile without, and the remaining 18.6% carried neither marker in that window and were not inspected — whether they are validated in the handler body, where a regex cannot look, is unverified.
• sysvar-spoofing was wrong rather than noisy: Sysvar::from_account_info validates the account key itself, so the call it flagged cannot be spoofed.
• unsafe-type-cast was removed before that run, on separate evidence: 73 of 110 hits, 66% of output over the 173-target eval corpus, with no confirmed true positive; it cannot see the source type, so it flags safe widening.

What remains is honest but weak, and the residue says the same thing the removals did: 21 of the 22 are non-canonical-bump, and reading 12 of those 21 by hand shows the legitimate stored-canonical-bump re-derivation pattern, a test stub and a CLI tool; the other 9 were not examined. Separating that from an attacker-supplied bump requires knowing where the bump came from. Every rule that fired with volume failed the same way — it matched an idiom and could not see the adjacent context (#[account], the bump's origin, the handler body) that decides the answer. That is the case for building the deterministic engine in core/, not for more patterns.

Vulnerability knowledge & reporting

The analyzers work through a structured Solana vulnerability catalog (src/knowledge/solana-vulns.ts) — access-control, CPI, PDA, arithmetic, lifecycle, oracle, DeFi (slippage/front-running), availability (DoS), governance (upgrade authority), Token-2022 extensions, and business-logic classes — grounded in the Sealevel Attacks, Neodyme's common pitfalls, and the solsec corpus. Every finding is tagged with a catalog id and the set of evaluated classes is tracked as coverage.

Findings are rated on an impact × likelihood severity matrix (src/knowledge/severity.ts), and the REPORT phase emits a professional assessment — executive summary with a deterministic severity table, scope & methodology, stable finding IDs (ARES-001…), per-finding description/impact/recommendation, and a coverage section — in the style of firms like OtterSec, Neodyme, and Zellic.

Security

See SECURITY.md for the vulnerability-reporting process, ARES's read-only runtime posture, and the status of tracked dependency advisories.

Billing (credits)

src/billing/ turns ARES into a paid service. It's opt-in — with BILLING_ENABLED unset, nothing here runs. When enabled, every audit is metered on its LLM token usage, priced against a per-model rate table (pricing.ts, tracking Anthropic/OpenRouter rates incl. prompt-cache and web-search costs), marked up, and charged in credits (default 100 credits = $1) against a two-tier account:

• System credits — a prepaid balance (a subscription's allotment or a top-up), drawn down first.
• On-demand credits — pay-as-you-go overflow once the prepaid balance is exhausted, settled via the Machine Payments Protocol (HTTP-402 pay-per-request / metered sessions; hermetic local settlement unless MPP_ENDPOINT is set).

Profit is guaranteed by construction (profit.ts): the effective markup is floored at 1 + BILLING_MIN_MARGIN_PCT, credits are rounded up, and every audit is billed at least BILLING_MIN_CHARGE_CREDITS — so a run can never settle below provider cost, even if the markup is misconfigured. Each settlement reports cost, revenue, profit, and margin.

Balances are stored with a lock file and written atomically, and a run refuses to overwrite a balance another audit changed underneath it (ConcurrentAccountUpdateError) rather than silently erasing the other charge. A store file that exists but cannot be parsed is a fatal error, not an excuse to start from a fresh allotment.

Payment is enforced before delivery. When billing is on, the report is released only after settlement succeeds; if the account can't cover the charge (InsufficientCreditsError), the report is withheld and the run exits non-zero. A pre-flight check also warns when an account structurally can't pay (no prepaid balance and on-demand disabled). Balances persist across runs when BILLING_ACCOUNT_STORE_PATH is set (account-store.ts), so on-demand spend actually accumulates instead of resetting each run. And a configured MPP_ENDPOINT whose real HTTP-402 client isn't wired fails loudly rather than silently settling locally — set MPP_ALLOW_LOCAL_FALLBACK=true to opt into hermetic settlement explicitly.

Configuration

All variables are documented in .env.example. Only OPENROUTER_API_KEY is required; Solana defaults to mainnet-beta, Postgres to the docker-compose values, and Supabase/Neo4j/embeddings/Helius are optional enhancements.

Extending

• New analyzers: add a node under src/graph/nodes/, wire it into the ANALYZE fan-out in src/graph/build-graph.ts, and append Findings.
• New tools: add under src/tools/. Remote tools can be surfaced via the Model Context Protocol (MCP) with the same load/run/normalize shape.
• New knowledge sources: implement the Retriever interface (src/retrieval/types.ts) and add it to createHybridRetriever.