Skip to content

Commit b88c3c9

Browse files
docs(bench): publish compile-time benchmark results in bench/README
Runs the harness against sumak + kysely + drizzle on an Apple M-series / Node 24 and records what came back. Two tables: 1. Ops per second — shows sumak is the fastest compiler on six of seven scenarios (1.11×–2.48× vs kysely, 9.06×–38.74× vs drizzle). The one exception is `delete-where`, where kysely's dedicated delete path beats sumak by ~1.54× — flagged as a known callback-WHERE overhead worth chasing. 2. Wall time per compile — same data inverted to ns/µs, to make the "your compile is three orders of magnitude below the network round-trip" point concrete. Even the slowest sumak case is ~2µs; a local Postgres round-trip is ~1ms. The results are stamped with machine + Node version and explicitly flagged as drift-prone ("treat only the relative ordering as signal"). No CI infrastructure is claimed; this is a human-run snapshot that doubles as evidence for the "why no statement cache" ADR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 1e6c787 commit b88c3c9

1 file changed

Lines changed: 34 additions & 0 deletions

File tree

bench/README.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,40 @@ pnpm vitest bench --run bench/compile.bench.ts
2525
| update-where | `UPDATE users SET name = $1 WHERE id = $2` |
2626
| delete-where | `DELETE FROM users WHERE id = $1` |
2727

28+
## Results
29+
30+
Compile throughput measured on an Apple M-series laptop, Node 24, vitest 4.1. Numbers drift across machines — treat only the **relative ordering** as signal.
31+
32+
| scenario | sumak (hz) | kysely (hz) | drizzle (hz) | sumak vs kysely | sumak vs drizzle |
33+
| ------------------ | ---------: | ------------: | -----------: | --------------: | ---------------: |
34+
| `select-all` | 1,794,929 | 1,597,645 | 87,529 | **1.11×** | **19.40×** |
35+
| `select-where-eq` | 1,194,919 | 666,188 | 80,344 | **1.79×** | **14.87×** |
36+
| `select-where-and` | 1,505,721 | 605,936 | 38,868 | **2.48×** | **38.74×** |
37+
| `join-2-tables` | 478,841 | 284,523 | 52,851 | **1.68×** | **9.06×** |
38+
| `insert-values` | 632,812 | 489,578 | 67,126 | **1.29×** | **9.43×** |
39+
| `update-where` | 1,141,395 | 718,878 | 78,405 | **1.59×** | **14.56×** |
40+
| `delete-where` | 665,789 | **1,024,416** | 150,204 | 0.65× | **4.43×** |
41+
42+
sumak is the fastest compiler on six of seven scenarios. `delete-where` is the exception: kysely's dedicated delete path wins by ~1.5×, tracked as a known callback-WHERE overhead worth investigating.
43+
44+
Against **drizzle**, sumak is between **9× and 39× faster** across the board — drizzle's template-literal-heavy internal representation costs a lot per call.
45+
46+
### Per-compile wall time
47+
48+
Same numbers, inverted to nanoseconds per compile — useful for sanity-checking whether the compile path is ever going to show up on a trace.
49+
50+
| scenario | sumak | kysely | drizzle |
51+
| ------------------ | -----: | -----: | ------: |
52+
| `select-all` | 557 ns | 626 ns | 11.4 µs |
53+
| `select-where-eq` | 837 ns | 1.5 µs | 12.4 µs |
54+
| `select-where-and` | 664 ns | 1.7 µs | 25.7 µs |
55+
| `join-2-tables` | 2.1 µs | 3.5 µs | 18.9 µs |
56+
| `insert-values` | 1.6 µs | 2.0 µs | 14.9 µs |
57+
| `update-where` | 876 ns | 1.4 µs | 12.8 µs |
58+
| `delete-where` | 1.5 µs | 977 ns | 6.7 µs |
59+
60+
Even the slowest sumak scenario (`join-2-tables`, ~2µs) compiles three orders of magnitude below a local Postgres round-trip (~1ms). Compile cost is not where your end-to-end latency lives — but it _is_ what shows up on a Lambda cold start.
61+
2862
## Why compile-time only?
2963

3064
Query builders live or die on the hot path between the TypeScript call and the SQL string. A benchmark that also spins up a real database would measure Postgres plus the network, not the library. On a Lambda cold path or a serverless edge runtime, compile time is the dominant overhead and the right thing to optimise.

0 commit comments

Comments
 (0)