Skip to content

INDEX.md

Latest commit

 

History

History
86 lines (67 loc) · 6.85 KB

INDEX.md

File metadata and controls

86 lines (67 loc) · 6.85 KB
Raw

bee-bench — results landing page

Cross-client benchmark of three Swarm Bee API clients: bee-go, bee-rs, bee-js.

⚠ MB/s numbers are NOT Swarm-network throughput

Every byte/sec figure produced by this bench is client ↔ local Bee node over loopback HTTP, NOT real Swarm-network throughput.

  • Uploads measure how fast the client pushes data into the local Bee node's store under deferred-upload mode (Swarm-Deferred-Upload: true, the Bee server-side default). The chunks are still being pushed to the Swarm network in the background after the call returns.
  • Downloads of references uploaded earlier in the same run hit the local Bee cache. Numbers like "253 MB/s" are local-store reads, not network fetches.

See FINDINGS § measurement scope for how to re-run with deferred: false for real-network numbers.

Where to look

Doc What's inside
Live HTML report Interactive Chart.js view served from GitHub Pages — open directly in a browser.
Latest report (markdown) Auto-generated per-case tables, scoreboard, sparklines, latency-vs-size linear fits. The numerical truth.
Latest report (HTML) Same as the live report above, but the raw HTML file in this repo.
FINDINGS Hand-written qualitative observations (F1–F21) — what the numbers actually mean, where the gaps come from, what to investigate.
README How to run the bench, env vars, layout.
bench-spec.json Source of truth for cases, sizes, iter counts.
aggregate.json Machine-readable form of report.md. Diff two snapshots with scripts/compare.mjs.
report.csv Flat CSV — one row per (case, param, runner). For spreadsheet / pandas analysis. Regenerated by node scripts/export-csv.mjs.

Quick highlights

From the latest run (see report.md § Scoreboard for the full table):

Runner Wins Overall ratio CPU HTTP-stack (Calibration + Pin)
bee-rs 24 1.11x 1.26x 1.00x
bee-go 10 1.32x 1.15x 6.04x (Calibration), 2.35x (Pin)
bee-js 0 4.48x 30.9x 14.5x (Calibration), 2.58x (Pin)
  • bee-rs leads on every group except CPU in the snapshot above (which is against bee-rs 1.1.0); the bee-rs 1.2.0 secp256k1 swap (2026-05-04) flips CPU as well — sign 3.57×, verify 4.50×, identity 2.68× (F5 / F15 / F17)
  • bee-js is 947x slower than bee-go on ECDSA verify (F15)
  • bee-go is unexpectedly 2.93x slower on stream-dir uploads (F19)
  • HTTP-stack pattern: reqwest pools connections better than Go's http.DefaultClient and far better than axios's keepAlive=false default (F18)

Findings index

Direct links to each finding in ../FINDINGS.md:

Crypto / CPU:

HTTP / network:

Client-API gaps:

Snapshots

Snapshots of earlier runs are preserved locally by ./scripts/preserve-run.sh <label> into results/<label>/. They contain their own aggregate.json / report.md / report.html plus the raw per-runner JSONs. They are not pushed to the remote — the per-runner JSONs are large and the canonical view here is the latest aggregate. To see a snapshot, regenerate it locally with ./scripts/run-all.sh + node scripts/aggregate.mjs.

Comparing two snapshots

Once you have two aggregate.json files locally:

node scripts/compare.mjs \
  results/<old-snapshot>/aggregate.json \
  results/<new-snapshot>/aggregate.json \
  --out results/compare.md

Compare report shows per-runner geomean shift and per-row deltas, flagging anything > ±20% with ⚠.