diff --git a/CMakeLists.txt b/CMakeLists.txt index 83d5b0b..1dd32e0 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -42,6 +42,11 @@ target_link_libraries(qsl-export-fixture PRIVATE qsl_core qsl_warnings) add_executable(qsl-export-stream apps/qsl-export-stream/main.cpp) target_link_libraries(qsl-export-stream PRIVATE qsl_core qsl_warnings) +# Performance-evidence harness (PERFORMANCE.md). Separate binary: it overrides operator new to count +# allocations, which must not perturb the qsl-bench numbers. Built Release for honest measurement. +add_executable(qsl-perfeval apps/qsl-perfeval/main.cpp) +target_link_libraries(qsl-perfeval PRIVATE qsl_core qsl_warnings) + if(QSL_BUILD_BENCHMARKS) find_package(Threads REQUIRED) add_executable(qsl-bench apps/qsl-bench/main.cpp benchmarks/bench_order_pool.cpp diff --git a/PERFORMANCE.md b/PERFORMANCE.md new file mode 100644 index 0000000..3e93355 --- /dev/null +++ b/PERFORMANCE.md @@ -0,0 +1,162 @@ +# Performance Evidence — matching-engine hot path + +This is the performance-evidence report for the v0.2.2 order-book optimizations. It profiles the +matching-engine hot path with Linux `perf` and flamegraphs on **ARM64 (Apple M2, Fedora Asahi)**, +identifies **order-book insertion and matching as the dominant cost**, and documents the +**before → after** change in latency, throughput, and CPU counters. Every number comes from the +committed `qsl-perfeval` harness and `perf`; nothing is estimated. + +> Scope and honesty. This is a single-machine, single-process, synthetic micro-evidence report — +> not a production-latency or HFT-readiness claim. Absolute numbers are hardware/compiler/build/ +> thermal dependent; the **before/after delta** measured back-to-back on the same host is the load- +> bearing result. Two metrics are reported honestly as **unavailable** rather than estimated: +> cache-miss counters (the Apple Silicon PMU does not expose them — [issue #90]) and any +> sub-`steady_clock`-resolution timing. + +## Optimizations under test + +| # | Change | Where | +|---|---|---| +| #138 | `std::map::emplace` → `try_emplace` for baseline price levels | `OrderBook::level_for` | +| #145 | order-index `unordered_map` `max_load_factor` 1.0 → **0.25** | `OrderBook` constructor | + +Both preserve determinism (the differential fixtures are byte-identical across g++/clang++ and vs +the committed copies; the OCaml differential passes). The index is never iterated for output, so +changing its bucket count cannot change emitted events or snapshots. + +## Headline before/after + +Workload: `qsl-perfeval 60000000` — a steady-state deep book (~512 resting orders, baseline storage). +Each **order** is one `new_limit` (it may match resting liquidity and rest its remainder); the book +is held ~512 deep by cancelling the oldest order each cycle, so the per-order throughput cost +includes that maintenance cancel. + +| Metric | Before | After | Δ | +|---|---|---|---| +| **Throughput** (orders/sec) | 8.89 M | **11.13 M** | **+25.2 %** | +| Median (p50) latency¹ | 83 ns | 83 ns | ~0 | +| **p99 latency**¹ | 250 ns | **208 ns** | **−16.8 %** | +| Mean latency¹ | 92 ns | 75 ns | −18.5 % | +| **Cycles / order** | 348.2 | **288.4** | **−17.2 %** | +| Instructions / order | 1239 | 1143 | −7.8 % | +| IPC | 3.56 | 3.96 | +11.4 % | +| Branches / order | 244 | 229 | −6.1 % | +| **Branch-miss rate** | 2.02 % | **1.81 %** | −0.21 pp | +| Cache-miss rate | _unavailable_ | _unavailable_ | — ([#90]) | +| **Allocations / order** | 1.106 | 1.106 | **0 (unchanged)** | + +¹ Latency is per `new_limit` only (not the maintenance cancel) and includes ~12 ns of `steady_clock` +read overhead per measured op (two VDSO clock reads); the before/after delta cancels it. Throughput +is per full cycle (`new_limit` + cancel), which is why p50 (~83 ns) is below the per-cycle wall time +(1 / 8.89 M ≈ 112 ns before; ≈ 90 ns after). + +### The honest mechanism + +The win is **fewer cycles and instructions per order**, not fewer allocations: + +- **Allocations are unchanged** (1.106 → 1.106). The original `#138` rationale ("`emplace` allocates + then frees a throwaway map node") turned out to be **wrong for libstdc++** — `std::map::emplace` + checks the key *before* allocating a node, so it does not churn the heap. The `try_emplace` win is + avoiding the construction/destruction of a throwaway empty `std::pmr::list` (and the heavier + `emplace` code path) on every insert when the level already exists — pure instruction savings, + which the counters confirm. This correction is the whole point of measuring with hardware counters + instead of guessing. +- **Shorter index probe chains.** Capping the order-index load factor at 0.25 keeps the + `OrderId → Locator` hash table sparse, so each of the 1–4 lookups per order (`contains`, `cancel` + find/erase, `rest` insert) probes fewer buckets. That shows up as lower instructions/order **and** + higher IPC (+11.4 %) — shorter chains stall the pipeline/memory system less — and a lower + branch-miss rate (fewer mispredicted bucket-traversal loop branches). The cache-locality component + is plausible but **not directly measurable here** (no cache counters; [#90]). + +## Profiling: where the time goes + +`perf record --call-graph fp` on `qsl-perfeval`, rendered with the dependency-free +`scripts/flamegraph.py` (no external FlameGraph toolkit). Frame width ∝ on-CPU samples. + +| Before | After | +|---|---| +| [![before](docs/performance/before.svg)](docs/performance/before.svg) | [![after](docs/performance/after.svg)](docs/performance/after.svg) | + +`perf report` (children %, hot path) confirms **order-book insertion + matching dominate**, and pins +the two optimizations' effect: + +``` + BEFORE AFTER +MatchingEngine::new_limit 80.1 % 83.2 % + OrderBook::add_limit 69.5 % 74.7 % + OrderBook::match_baseline 25.7 % 32.0 % <- matching + OrderBook::rest 33.3 % 31.8 % <- insertion + OrderBook::level_for 21.3 % -> 17.5 % <- #138 try_emplace + OrderBook::contains 3.6 % -> 1.3 % <- #145 load-factor (dup-id lookup) +MatchingEngine::cancel 18.2 % 15.8 % + OrderBook::cancel 16.0 % -> 13.2 % <- #145 load-factor (find + erase) +``` + +(Percentages are of total samples, so as the optimized functions shrink the survivors grow +proportionally — e.g. `new_limit` rises 80→83 % only because the total dropped. The *absolute* wins +are `level_for`, `contains`, and `cancel` all falling, exactly the two changes' targets.) + +`perf annotate` attributes the remaining cost of `level_for` to the `std::_Rb_tree` lookup/insert +loads (`ldr`/`ldp` over the red-black-tree nodes) — the inherent cost of an ordered price-level map, +not avoidable allocation. + +## Hardware counters + +Full raw `perf stat` for both builds, with derivations and the counter-availability caveat, is in +**[`docs/performance/perf-stat.txt`](docs/performance/perf-stat.txt)**. Cycles, instructions, +branches, and branch-misses are **real Apple Avalanche P-core PMU counts**; cache-references / +cache-misses are **not implemented** by this PMU ([#90]), so cache-miss rate is reported as +unavailable, never estimated. + +## Methodology / reproduction + +``` +Hardware Apple M2 (aarch64), Avalanche performance cores (MIDR CPU part 0x032), bare metal +Kernel Linux 6.19.14-400.asahi.fc44.aarch64+16k (Fedora Asahi Remix) +Governor schedutil +Compiler GCC (c++) 16.1.1 +Flags Release (-O3 -DNDEBUG) + -fno-omit-frame-pointer -g (CMake "flamegraph" preset) +perf 6.19.14, kernel.perf_event_paranoid = 2 +``` + +Reproduce (the `qsl-perfeval` harness is a dedicated binary — it overrides global `operator new` to +count allocations, kept out of `qsl-bench` so it cannot perturb `results/latest.txt`): + +```bash +cmake --preset flamegraph +cmake --build --preset flamegraph --target qsl-perfeval +BIN=build/flamegraph/qsl-perfeval + +# throughput + allocations/order (clean: no per-op timer in the cycle count) +"$BIN" 60000000 + +# latency distribution (mean / p50 / p99; includes timer overhead, reported) +"$BIN" 5000000 --latency + +# hardware counters +perf stat -e cycles,instructions,branches,branch-misses -- "$BIN" 60000000 + +# flamegraph +perf record --call-graph fp -F 4000 -g -e cpu-clock -o perf.data -- "$BIN" 60000000 +perf script -i perf.data | python3 scripts/flamegraph.py --collapse-only \ + | python3 scripts/flamegraph.py --from-collapsed > flame.svg + +# hot path / annotation +perf report -i perf.data --stdio +perf annotate -i perf.data --stdio +``` + +The **before** build is the same source with the two changes reverted (`emplace` in `level_for`, +no `max_load_factor` call). Before and after were measured back-to-back on the same host. + +## Tuning balance (why 0.25, not lower) + +The index load factor was swept on the profile workload: 0.5 → ~+10 %, 0.25 → ~+18 %, 0.125 → ~+20 %. +The curve flattens below ~0.25, so **0.25** captures essentially all of the throughput win as a clean +load-factor *policy* (memory scales with book size) rather than over-tuning a fixed bucket count or +paying 8× buckets-to-orders for the last ~2 %. Combined with `try_emplace` (an instruction-level win +with no memory cost), this is the minmax point: most of the available speed for a modest, principled +memory trade, with the hot path now bounded by the inherent red-black-tree price-level lookups and +the hash-index probes that any correct implementation must pay. + +[#90]: https://github.com/div0rce/quant-systems-lab/issues/90 diff --git a/apps/qsl-perfeval/main.cpp b/apps/qsl-perfeval/main.cpp new file mode 100644 index 0000000..cea2e6f --- /dev/null +++ b/apps/qsl-perfeval/main.cpp @@ -0,0 +1,242 @@ +// qsl-perfeval — a dedicated performance-evidence harness for the matching-engine hot path +// (order-book insertion + matching). It drives a steady-state deep-book order flow and reports +// orders/sec, per-order latency distribution (mean/p50/p99), and allocations/order. Run it under +// `perf stat` / `perf record` to attribute cycles/instructions/branch-misses and render +// flamegraphs. +// +// It is a SEPARATE binary from qsl-bench on purpose: it overrides global operator new/delete to +// count allocations, which must not perturb the committed qsl-bench numbers in results/latest.txt. +// +// Workload: each "order" is one new_limit submission (may match resting liquidity and rest its +// remainder); the book is held ~kRing deep by cancelling the oldest resting order each cycle. This +// matches the `qsl-bench profile` flamegraph workload's steady-state character. Baseline storage. + +#include "qsl/engine/matching_engine.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { +// Allocations during the timed region only (the counter is sampled before and after the loop). The +// override below counts the primary std::operator new path used by the baseline pmr new_delete +// resource for order-book list/map nodes. +std::atomic g_allocs{0}; +} // namespace + +void *operator new(std::size_t n) { + g_allocs.fetch_add(1, std::memory_order_relaxed); + if (void *p = std::malloc(n)) { + return p; + } + throw std::bad_alloc(); +} +void operator delete(void *p) noexcept { + std::free(p); +} +void operator delete(void *p, std::size_t) noexcept { + std::free(p); +} +void *operator new[](std::size_t n) { + g_allocs.fetch_add(1, std::memory_order_relaxed); + if (void *p = std::malloc(n)) { + return p; + } + throw std::bad_alloc(); +} +void operator delete[](void *p) noexcept { + std::free(p); +} +void operator delete[](void *p, std::size_t) noexcept { + std::free(p); +} + +namespace { +using clk = std::chrono::steady_clock; +using namespace qsl; + +constexpr std::size_t kRing = 512; // bounded resting depth +constexpr core::Price kBase = 100; +constexpr core::Price kBand = 64; // price band [100, 164) + +std::uint64_t splitmix64(std::uint64_t &s) { + s += 0x9E3779B97F4A7C15ULL; + std::uint64_t z = s; + z = (z ^ (z >> 30)) * 0xBF58476D1CE4E5B9ULL; + z = (z ^ (z >> 27)) * 0x94D049BB133111EBULL; + return z ^ (z >> 31); +} + +std::uint32_t ns_between(clk::time_point a, clk::time_point b) { + return static_cast( + std::chrono::duration_cast(b - a).count()); +} + +// Measured cost of two steady_clock reads around nothing, so the report can state how much of the +// per-order latency is timer overhead rather than engine work. +std::uint32_t timer_overhead_ns() { + constexpr int kIters = 200000; + std::uint64_t sum = 0; + for (int i = 0; i < kIters; ++i) { + const auto a = clk::now(); + const auto b = clk::now(); + sum += ns_between(a, b); + } + return static_cast(sum / kIters); +} + +struct PerfFlow { + engine::MatchingEngine eng{}; // baseline storage + core::SymbolId sym = eng.register_symbol("AAPL"); + std::vector ring{}; + std::size_t head = 0; + core::OrderId id = 1; + std::uint64_t state = 0x9E3779B97F4A7C15ULL; + volatile std::uint64_t sink = 0; + + PerfFlow() { ring.reserve(kRing); } + + // Submit one limit order; returns its id so the caller can park it in the ring. + core::OrderId submit() { + const std::uint64_t r = splitmix64(state); + const auto side = ((r & 1U) != 0U) ? core::Side::Buy : core::Side::Sell; + const auto price = kBase + static_cast((r >> 1) % kBand); + const auto qty = 1 + static_cast((r >> 8) % 8); + const core::OrderId oid = id++; + sink += eng.new_limit(sym, oid, side, price, qty, core::TimeInForce::GTC).size(); + return oid; + } + + // Cancel the oldest resting order and park the new id in its slot, holding the book ~kRing + // deep. + void retire_oldest(core::OrderId oid) { + sink += eng.cancel(sym, ring[head]).size(); + ring[head] = oid; + head = (head + 1) % kRing; + } + + void warmup() { + while (ring.size() < kRing) { + ring.push_back(submit()); + } + } +}; + +struct Result { + std::uint64_t orders = 0; + double seconds = 0.0; + double allocs_per_order = 0.0; + bool has_latency = false; + std::uint32_t mean_ns = 0, p50_ns = 0, p99_ns = 0, overhead_ns = 0; +}; + +void fill_latency_stats(std::vector &lat, Result &res) { + if (lat.empty()) { + return; + } + std::sort(lat.begin(), lat.end()); + std::uint64_t sum = 0; + for (const std::uint32_t v : lat) { + sum += v; + } + res.mean_ns = static_cast(sum / lat.size()); + res.p50_ns = lat[lat.size() / 2]; + res.p99_ns = lat[(lat.size() * 99) / 100]; + res.overhead_ns = timer_overhead_ns(); +} + +// Untimed run: cleanest for `perf stat`/`perf record` (no per-op clock overhead in the cycle +// count). +Result run_throughput(std::uint64_t orders) { + PerfFlow flow; + flow.warmup(); + const std::uint64_t a0 = g_allocs.load(std::memory_order_relaxed); + const auto t0 = clk::now(); + for (std::uint64_t i = 0; i < orders; ++i) { + flow.retire_oldest(flow.submit()); + } + const auto t1 = clk::now(); + Result res; + res.orders = orders; + res.seconds = std::chrono::duration(t1 - t0).count(); + res.allocs_per_order = static_cast(g_allocs.load(std::memory_order_relaxed) - a0) / + static_cast(orders); + return res; +} + +// Timed run: records per-order new_limit latency. Absolute values include steady_clock overhead +// (reported separately); the before/after delta is the meaningful signal. +Result run_latency(std::uint64_t orders) { + PerfFlow flow; + flow.warmup(); + std::vector lat; + lat.reserve(orders); + const std::uint64_t a0 = g_allocs.load(std::memory_order_relaxed); + const auto t0 = clk::now(); + for (std::uint64_t i = 0; i < orders; ++i) { + const std::uint64_t r = splitmix64(flow.state); + const auto side = ((r & 1U) != 0U) ? core::Side::Buy : core::Side::Sell; + const auto price = kBase + static_cast((r >> 1) % kBand); + const auto qty = 1 + static_cast((r >> 8) % 8); + const core::OrderId oid = flow.id++; + const auto a = clk::now(); + flow.sink += + flow.eng.new_limit(flow.sym, oid, side, price, qty, core::TimeInForce::GTC).size(); + const auto b = clk::now(); + lat.push_back(ns_between(a, b)); + flow.retire_oldest(oid); + } + const auto t1 = clk::now(); + Result res; + res.orders = orders; + res.seconds = std::chrono::duration(t1 - t0).count(); + res.allocs_per_order = static_cast(g_allocs.load(std::memory_order_relaxed) - a0) / + static_cast(orders); + res.has_latency = true; + fill_latency_stats(lat, res); + return res; +} + +std::uint64_t parse_orders(int argc, char **argv, bool latency) { + for (int i = 1; i < argc; ++i) { + const std::string a = argv[i]; + if (a != "--latency") { + const std::uint64_t n = std::strtoull(a.c_str(), nullptr, 10); + if (n > 0) { + return n; + } + } + } + return latency ? 5'000'000ULL : 60'000'000ULL; +} +} // namespace + +// qsl-perfeval [orders] [--latency] +// default: untimed throughput + allocations/order (run under perf stat / perf record) +// --latency: also record per-order latency mean/p50/p99 +int main(int argc, char **argv) { + bool latency = false; + for (int i = 1; i < argc; ++i) { + if (std::string(argv[i]) == "--latency") { + latency = true; + } + } + const std::uint64_t orders = parse_orders(argc, argv, latency); + const Result r = latency ? run_latency(orders) : run_throughput(orders); + std::printf("perfeval: storage=baseline orders=%llu elapsed=%.3fs orders_per_sec=%.0f " + "allocs_per_order=%.4f\n", + static_cast(r.orders), r.seconds, + static_cast(r.orders) / r.seconds, r.allocs_per_order); + if (r.has_latency) { + std::printf("perfeval: latency_ns mean=%u p50=%u p99=%u (each incl ~%uns timer overhead)\n", + r.mean_ns, r.p50_ns, r.p99_ns, r.overhead_ns); + } + return 0; +} diff --git a/docs/performance/after.svg b/docs/performance/after.svg new file mode 100644 index 0000000..7552c32 --- /dev/null +++ b/docs/performance/after.svg @@ -0,0 +1,15 @@ + +QSL matching-engine hot path — AFTER (try_emplace + index load-factor 0.25)qsl-perfeval steady-state deep book, baseline storage, aarch64 Apple M2, Release+fpSearch all (22046 samples, 100.00%)allqsl-perfeval (22046 samples, 100.00%)qsl-perfeval_start (22046 samples, 100.00%)_start__libc_start_main@@GLIBC_2.34 (22044 samples, 99.99%)__libc_start_main@@GLIBC_2.34__libc_start_call_main (22044 samples, 99.99%)__libc_start_call_mainmain (22044 samples, 99.99%)maincfree@GLIBC_2.17 (63 samples, 0.29%)qsl::engine::MatchingEngine::cancel(unsigned int, unsigned long) (3487 samples, 15.82%)qsl::engine::MatchingEn..qsl::engine::OrderBook::cancel(unsigned long) (2907 samples, 13.19%)qsl::engine::OrderB..decltype(auto) qsl::engine::OrderBook::dispatch_storage<qsl::engine::OrderBook::cancel(unsigned long)::{lambda()#1}, qsl::engine::OrderBook::cancel(unsigned long)::{lambda(qsl::engine::OrderBook::IntrusiveStore&)#1}, qsl::engine::OrderBook::cancel(unsigned long)::{lambda(qsl::engine::OrderBook::ContiguousStore&)#1}>(qsl::engine::OrderBook::cancel(unsigned long)::{lambda()#1}&&, qsl::engine::OrderBook::cancel(unsigned long)::{lambda(qsl::engine::OrderBook::IntrusiveStore&)#1}&&, qsl::engine::OrderBook::cancel(unsigned long)::{lambda(qsl::engine::OrderBook::ContiguousStore&)#1}&&) [clone .isra.0] (2826 samples, 12.82%)decltype(auto) qsl..cfree@GLIBC_2.17 (63 samples, 0.29%)qsl::engine::OrderBook::erase_resting_order(qsl::engine::OrderBook::Locator const&) (1334 samples, 6.05%)qsl::en.._int_free_chunk (9 samples, 0.04%)_int_free_merge_chunk (7 samples, 0.03%)_int_free_create_chunk (3 samples, 0.01%)cfree@GLIBC_2.17 (122 samples, 0.55%)free@plt (38 samples, 0.17%)operator delete(void*, std::align_val_t)@plt (30 samples, 0.14%)operator delete(void*, unsigned long, std::align_val_t)@plt (28 samples, 0.13%)std::_Rb_tree_rebalance_for_erase(std::_Rb_tree_node_base*, std::_Rb_tree_node_base&) (23 samples, 0.10%)std::__detail::_List_node_base::_M_unhook() (28 samples, 0.13%)std::__detail::_List_node_base::_M_unhook()@plt (32 samples, 0.15%)std::_Hashtable<unsigned long, std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, std::pmr::polymorphic_allocator<std::pair<unsigned long const, qsl::engine::OrderBook::Locator> >, std::__detail::_Select1st, std::equal_to<unsigned long>, std::hash<unsigned long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_erase(unsigned long, std::__detail::_Hash_node_base*, std::__detail::_Hash_node<std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, false>*) (348 samples, 1.58%)_int_free_chunk (18 samples, 0.08%)_int_free_maybe_trim (2 samples, 0.01%)_int_free_merge_chunk (11 samples, 0.05%)_int_free_create_chunk (7 samples, 0.03%)unlink_chunk.isra.0 (2 samples, 0.01%)cfree@GLIBC_2.17 (110 samples, 0.50%)free@plt (27 samples, 0.12%)operator delete(void*, std::align_val_t)@plt (32 samples, 0.15%)operator delete(void*, unsigned long, std::align_val_t)@plt (47 samples, 0.21%)std::pmr::(anonymous namespace)::newdel_res_t::do_deallocate(void*, unsigned long, unsigned long) (24 samples, 0.11%)qsl::engine::MatchingEngine::new_limit(unsigned int, unsigned long, qsl::core::Side, long, unsigned int, qsl::core::TimeInForce) (18348 samples, 83.23%)qsl::engine::MatchingEngine::new_limit(unsigned int, unsigned long, qsl::core::Side, long, unsigned int, qsl::core::TimeInForce)_int_free_chunk (10 samples, 0.05%)_int_free_merge_chunk (6 samples, 0.03%)_int_free_create_chunk (3 samples, 0.01%)cfree@GLIBC_2.17 (244 samples, 1.11%)free@plt (46 samples, 0.21%)qsl::engine::OrderBook::add_limit(unsigned long, qsl::core::Side, long, unsigned int, qsl::core::TimeInForce) (16467 samples, 74.69%)qsl::engine::OrderBook::add_limit(unsigned long, qsl::core::Side, long, unsigned int, qsl::core::TimeInForce)__memcpy_generic (174 samples, 0.79%)_int_free_chunk (2 samples, 0.01%)cfree@GLIBC_2.17 (503 samples, 2.28%)decltype(auto) qsl::engine::OrderBook::dispatch_storage<qsl::engine::OrderBook::contains(unsigned long) const::{lambda()#1}, qsl::engine::OrderBook::contains(unsigned long) const::{lambda(qsl::engine::OrderBook::IntrusiveStore const&)#1}, qsl::engine::OrderBook::contains(unsigned long) const::{lambda(qsl::engine::OrderBook::ContiguousStore const&)#1}>(qsl::engine::OrderBook::contains(unsigned long) const::{lambda()#1}&&, qsl::engine::OrderBook::contains(unsigned long) const::{lambda(qsl::engine::OrderBook::IntrusiveStore const&)#1}&&, qsl::engine::OrderBook::contains(unsigned long) const::{lambda(qsl::engine::OrderBook::ContiguousStore const&)#1}&&) const [clone .isra.0] (340 samples, 1.54%)free@plt (97 samples, 0.44%)memcpy@plt (31 samples, 0.14%)operator delete(void*, unsigned long) (10 samples, 0.05%)operator new(unsigned long) (450 samples, 2.04%)__aarch64_ldadd8_relax (95 samples, 0.43%)__libc_malloc2 (34 samples, 0.15%)_int_malloc (30 samples, 0.14%)malloc (228 samples, 1.03%)malloc@plt (24 samples, 0.11%)qsl::engine::OrderBook::match_baseline(qsl::core::Side, qsl::engine::OrderBook::MatchContext&) (7061 samples, 32.03%)qsl::engine::OrderBook::match_baseline(qsl::core::S..__memcpy_generic (61 samples, 0.28%)_int_free_chunk (43 samples, 0.20%)_int_free_maybe_trim (4 samples, 0.02%)_int_free_merge_chunk (32 samples, 0.15%)_int_free_create_chunk (19 samples, 0.09%)unlink_chunk.isra.0 (3 samples, 0.01%)unlink_chunk.isra.0 (2 samples, 0.01%)cfree@GLIBC_2.17 (465 samples, 2.11%)free@plt (103 samples, 0.47%)memcpy@plt (17 samples, 0.08%)operator delete(void*, std::align_val_t)@plt (89 samples, 0.40%)operator delete(void*, unsigned long, std::align_val_t)@plt (124 samples, 0.56%)operator new(unsigned long) (323 samples, 1.47%)__aarch64_ldadd8_relax (88 samples, 0.40%)__libc_malloc2 (14 samples, 0.06%)_int_malloc (13 samples, 0.06%)malloc (153 samples, 0.69%)malloc@plt (18 samples, 0.08%)qsl::engine::OrderBook::fill_front_order(std::__cxx11::list<qsl::engine::Order, std::pmr::polymorphic_allocator<qsl::engine::Order> >&, long, qsl::engine::OrderBook::MatchContext&) (1816 samples, 8.24%)qsl::engine..__memcpy_generic (53 samples, 0.24%)_int_free_chunk (30 samples, 0.14%)_int_free_maybe_trim (6 samples, 0.03%)_int_free_merge_chunk (19 samples, 0.09%)_int_free_create_chunk (11 samples, 0.05%)unlink_chunk.isra.0 (3 samples, 0.01%)cfree@GLIBC_2.17 (268 samples, 1.22%)free@plt (40 samples, 0.18%)memcpy@plt (8 samples, 0.04%)operator delete(void*, std::align_val_t)@plt (28 samples, 0.13%)operator delete(void*, unsigned long, std::align_val_t)@plt (55 samples, 0.25%)operator new(unsigned long) (358 samples, 1.62%)__aarch64_ldadd8_relax (76 samples, 0.34%)__libc_malloc2 (12 samples, 0.05%)_int_malloc (11 samples, 0.05%)malloc (174 samples, 0.79%)malloc@plt (23 samples, 0.10%)std::_Hashtable<unsigned long, std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, std::pmr::polymorphic_allocator<std::pair<unsigned long const, qsl::engine::OrderBook::Locator> >, std::__detail::_Select1st, std::equal_to<unsigned long>, std::hash<unsigned long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_erase(unsigned long, std::__detail::_Hash_node_base*, std::__detail::_Hash_node<std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, false>*) (333 samples, 1.51%)_int_free_chunk (14 samples, 0.06%)_int_free_merge_chunk (9 samples, 0.04%)_int_free_create_chunk (6 samples, 0.03%)_int_free_maybe_trim (2 samples, 0.01%)cfree@GLIBC_2.17 (111 samples, 0.50%)free@plt (29 samples, 0.13%)operator delete(void*, std::align_val_t)@plt (30 samples, 0.14%)operator delete(void*, unsigned long, std::align_val_t)@plt (42 samples, 0.19%)std::pmr::(anonymous namespace)::newdel_res_t::do_deallocate(void*, unsigned long, unsigned long) (18 samples, 0.08%)std::__detail::_List_node_base::_M_unhook() (49 samples, 0.22%)std::__detail::_List_node_base::_M_unhook()@plt (15 samples, 0.07%)std::pmr::(anonymous namespace)::newdel_res_t::do_deallocate(void*, unsigned long, unsigned long) (24 samples, 0.11%)std::_Hashtable<unsigned long, std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, std::pmr::polymorphic_allocator<std::pair<unsigned long const, qsl::engine::OrderBook::Locator> >, std::__detail::_Select1st, std::equal_to<unsigned long>, std::hash<unsigned long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_erase(unsigned long, std::__detail::_Hash_node_base*, std::__detail::_Hash_node<std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, false>*) (387 samples, 1.76%)_int_free_chunk (17 samples, 0.08%)_int_free_maybe_trim (3 samples, 0.01%)_int_free_merge_chunk (12 samples, 0.05%)_int_free_create_chunk (7 samples, 0.03%)cfree@GLIBC_2.17 (134 samples, 0.61%)free@plt (27 samples, 0.12%)operator delete(void*, std::align_val_t)@plt (27 samples, 0.12%)operator delete(void*, unsigned long, std::align_val_t)@plt (49 samples, 0.22%)std::pmr::(anonymous namespace)::newdel_res_t::do_deallocate(void*, unsigned long, unsigned long) (28 samples, 0.13%)std::_Rb_tree_rebalance_for_erase(std::_Rb_tree_node_base*, std::_Rb_tree_node_base&) (714 samples, 3.24%)st..std::_Rb_tree_rebalance_for_erase(std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)@plt (19 samples, 0.09%)std::__detail::_List_node_base::_M_unhook() (49 samples, 0.22%)std::__detail::_List_node_base::_M_unhook()@plt (12 samples, 0.05%)std::pmr::(anonymous namespace)::newdel_res_t::do_deallocate(void*, unsigned long, unsigned long) (59 samples, 0.27%)qsl::engine::OrderBook::rest(unsigned long, qsl::core::Side, long, unsigned int) (7010 samples, 31.80%)qsl::engine::OrderBook::rest(unsigned long, qsl::c..__posix_memalign (9 samples, 0.04%)operator new(unsigned long, std::align_val_t) (921 samples, 4.18%)oper..__posix_memalign (663 samples, 3.01%)__..__libc_malloc2 (104 samples, 0.47%)_int_malloc (87 samples, 0.39%)unlink_chunk.isra.0 (4 samples, 0.02%)_mid_memalign (94 samples, 0.43%)malloc (279 samples, 1.27%)_mid_memalign (37 samples, 0.17%)posix_memalign@plt (54 samples, 0.24%)operator new(unsigned long, std::align_val_t)@plt (130 samples, 0.59%)qsl::engine::OrderBook::level_for[abi:cxx11](qsl::core::Side, long) (3854 samples, 17.48%)qsl::engine::OrderBook::le..operator new(unsigned long, std::align_val_t) (409 samples, 1.86%)__posix_memalign (269 samples, 1.22%)__libc_malloc2 (2 samples, 0.01%)_int_malloc (2 samples, 0.01%)_mid_memalign (52 samples, 0.24%)malloc (127 samples, 0.58%)_mid_memalign (23 samples, 0.10%)posix_memalign@plt (29 samples, 0.13%)operator new(unsigned long, std::align_val_t)@plt (73 samples, 0.33%)std::_Rb_tree_decrement(std::_Rb_tree_node_base*) (112 samples, 0.51%)std::_Rb_tree_decrement(std::_Rb_tree_node_base*)@plt (8 samples, 0.04%)std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&) (571 samples, 2.59%)s..std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)@plt (51 samples, 0.23%)std::pmr::(anonymous namespace)::newdel_res_t::do_allocate(unsigned long, unsigned long) (24 samples, 0.11%)std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*) (71 samples, 0.32%)std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*)@plt (58 samples, 0.26%)std::__detail::_Map_base<unsigned long, std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, std::pmr::polymorphic_allocator<std::pair<unsigned long const, qsl::engine::OrderBook::Locator> >, std::__detail::_Select1st, std::equal_to<unsigned long>, std::hash<unsigned long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true>, true>::operator[](unsigned long const&) (1766 samples, 8.01%)std::__det..__posix_memalign (39 samples, 0.18%)operator new(unsigned long, std::align_val_t) (806 samples, 3.66%)ope..__posix_memalign (507 samples, 2.30%)_..__libc_malloc2 (20 samples, 0.09%)_int_malloc (16 samples, 0.07%)_int_malloc (2 samples, 0.01%)_mid_memalign (101 samples, 0.46%)malloc (256 samples, 1.16%)_mid_memalign (38 samples, 0.17%)posix_memalign@plt (52 samples, 0.24%)operator new(unsigned long, std::align_val_t)@plt (171 samples, 0.78%)std::_Hashtable<unsigned long, std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, std::pmr::polymorphic_allocator<std::pair<unsigned long const, qsl::engine::OrderBook::Locator> >, std::__detail::_Select1st, std::equal_to<unsigned long>, std::hash<unsigned long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_insert_unique_node(unsigned long, unsigned long, std::__detail::_Hash_node<std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, false>*, unsigned long) (410 samples, 1.86%)std::__detail::_Prime_rehash_policy::_M_need_rehash(unsigned long, unsigned long, unsigned long) const (162 samples, 0.73%)std::__detail::_Prime_rehash_policy::_M_need_rehash(unsigned long, unsigned long, unsigned long) const@plt (39 samples, 0.18%)std::pmr::(anonymous namespace)::newdel_res_t::do_allocate(unsigned long, unsigned long) (28 samples, 0.13%)std::pmr::(anonymous namespace)::newdel_res_t::do_allocate(unsigned long, unsigned long) (2 samples, 0.01%)qsl::engine::OrderBook::can_store_limit(qsl::core::Side, long, unsigned int, qsl::core::TimeInForce) const (198 samples, 0.90%)qsl::engine::OrderBook::contains(unsigned long) const (280 samples, 1.27%) diff --git a/docs/performance/before.svg b/docs/performance/before.svg new file mode 100644 index 0000000..4a0e422 --- /dev/null +++ b/docs/performance/before.svg @@ -0,0 +1,15 @@ + +QSL matching-engine hot path — BEFORE (std::map::emplace + default index load-factor)qsl-perfeval steady-state deep book, baseline storage, aarch64 Apple M2, Release+fpSearch all (26115 samples, 100.00%)allqsl-perfeval (26115 samples, 100.00%)qsl-perfeval_start (26115 samples, 100.00%)_start__libc_start_main@@GLIBC_2.34 (26114 samples, 100.00%)__libc_start_main@@GLIBC_2.34__libc_start_call_main (26114 samples, 100.00%)__libc_start_call_mainmain (26113 samples, 99.99%)maincfree@GLIBC_2.17 (36 samples, 0.14%)qsl::engine::MatchingEngine::cancel(unsigned int, unsigned long) (4811 samples, 18.42%)qsl::engine::MatchingEngine:..qsl::engine::OrderBook::cancel(unsigned long) (4178 samples, 16.00%)qsl::engine::OrderBook::..decltype(auto) qsl::engine::OrderBook::dispatch_storage<qsl::engine::OrderBook::cancel(unsigned long)::{lambda()#1}, qsl::engine::OrderBook::cancel(unsigned long)::{lambda(qsl::engine::OrderBook::IntrusiveStore&)#1}, qsl::engine::OrderBook::cancel(unsigned long)::{lambda(qsl::engine::OrderBook::ContiguousStore&)#1}>(qsl::engine::OrderBook::cancel(unsigned long)::{lambda()#1}&&, qsl::engine::OrderBook::cancel(unsigned long)::{lambda(qsl::engine::OrderBook::IntrusiveStore&)#1}&&, qsl::engine::OrderBook::cancel(unsigned long)::{lambda(qsl::engine::OrderBook::ContiguousStore&)#1}&&) [clone .isra.0] (4071 samples, 15.59%)decltype(auto) qsl::eng..cfree@GLIBC_2.17 (64 samples, 0.25%)qsl::engine::OrderBook::erase_resting_order(qsl::engine::OrderBook::Locator const&) (1381 samples, 5.29%)qsl::e.._int_free_chunk (10 samples, 0.04%)_int_free_merge_chunk (4 samples, 0.02%)cfree@GLIBC_2.17 (120 samples, 0.46%)free@plt (26 samples, 0.10%)operator delete(void*, std::align_val_t)@plt (34 samples, 0.13%)operator delete(void*, unsigned long, std::align_val_t)@plt (24 samples, 0.09%)std::_Rb_tree_rebalance_for_erase(std::_Rb_tree_node_base*, std::_Rb_tree_node_base&) (28 samples, 0.11%)std::__detail::_List_node_base::_M_unhook() (19 samples, 0.07%)std::__detail::_List_node_base::_M_unhook()@plt (38 samples, 0.15%)std::_Hashtable<unsigned long, std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, std::pmr::polymorphic_allocator<std::pair<unsigned long const, qsl::engine::OrderBook::Locator> >, std::__detail::_Select1st, std::equal_to<unsigned long>, std::hash<unsigned long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_erase(unsigned long, std::__detail::_Hash_node_base*, std::__detail::_Hash_node<std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, false>*) (362 samples, 1.39%)_int_free_chunk (16 samples, 0.06%)_int_free_merge_chunk (12 samples, 0.05%)_int_free_create_chunk (5 samples, 0.02%)cfree@GLIBC_2.17 (113 samples, 0.43%)free@plt (28 samples, 0.11%)operator delete(void*, std::align_val_t)@plt (28 samples, 0.11%)operator delete(void*, unsigned long, std::align_val_t)@plt (63 samples, 0.24%)std::pmr::(anonymous namespace)::newdel_res_t::do_deallocate(void*, unsigned long, unsigned long) (10 samples, 0.04%)qsl::engine::MatchingEngine::new_limit(unsigned int, unsigned long, qsl::core::Side, long, unsigned int, qsl::core::TimeInForce) (21082 samples, 80.73%)qsl::engine::MatchingEngine::new_limit(unsigned int, unsigned long, qsl::core::Side, long, unsigned int, qsl::core::TimeInForce)_int_free_chunk (8 samples, 0.03%)_int_free_merge_chunk (5 samples, 0.02%)_int_free_create_chunk (4 samples, 0.02%)cfree@GLIBC_2.17 (255 samples, 0.98%)free@plt (47 samples, 0.18%)qsl::engine::OrderBook::add_limit(unsigned long, qsl::core::Side, long, unsigned int, qsl::core::TimeInForce) (18595 samples, 71.20%)qsl::engine::OrderBook::add_limit(unsigned long, qsl::core::Side, long, unsigned int, qsl::core::TimeInForce)__memcpy_generic (176 samples, 0.67%)cfree@GLIBC_2.17 (427 samples, 1.64%)decltype(auto) qsl::engine::OrderBook::dispatch_storage<qsl::engine::OrderBook::contains(unsigned long) const::{lambda()#1}, qsl::engine::OrderBook::contains(unsigned long) const::{lambda(qsl::engine::OrderBook::IntrusiveStore const&)#1}, qsl::engine::OrderBook::contains(unsigned long) const::{lambda(qsl::engine::OrderBook::ContiguousStore const&)#1}>(qsl::engine::OrderBook::contains(unsigned long) const::{lambda()#1}&&, qsl::engine::OrderBook::contains(unsigned long) const::{lambda(qsl::engine::OrderBook::IntrusiveStore const&)#1}&&, qsl::engine::OrderBook::contains(unsigned long) const::{lambda(qsl::engine::OrderBook::ContiguousStore const&)#1}&&) const [clone .isra.0] (365 samples, 1.40%)free@plt (111 samples, 0.43%)memcpy@plt (43 samples, 0.16%)operator delete(void*, unsigned long) (11 samples, 0.04%)operator new(unsigned long) (486 samples, 1.86%)__aarch64_ldadd8_relax (82 samples, 0.31%)__libc_malloc2 (38 samples, 0.15%)_int_malloc (32 samples, 0.12%)malloc (253 samples, 0.97%)malloc@plt (29 samples, 0.11%)operator new(unsigned long, std::align_val_t) (5 samples, 0.02%)qsl::engine::OrderBook::match_baseline(qsl::core::Side, qsl::engine::OrderBook::MatchContext&) (7339 samples, 28.10%)qsl::engine::OrderBook::match_baseline(qsl::..__memcpy_generic (61 samples, 0.23%)_int_free_chunk (43 samples, 0.16%)_int_free_merge_chunk (31 samples, 0.12%)_int_free_create_chunk (15 samples, 0.06%)unlink_chunk.isra.0 (3 samples, 0.01%)_int_free_merge_chunk (4 samples, 0.02%)cfree@GLIBC_2.17 (491 samples, 1.88%)free@plt (90 samples, 0.34%)memcpy@plt (17 samples, 0.07%)operator delete(void*, std::align_val_t)@plt (76 samples, 0.29%)operator delete(void*, unsigned long, std::align_val_t)@plt (115 samples, 0.44%)operator new(unsigned long) (342 samples, 1.31%)__aarch64_ldadd8_relax (71 samples, 0.27%)__libc_malloc2 (16 samples, 0.06%)_int_malloc (15 samples, 0.06%)malloc (162 samples, 0.62%)malloc@plt (33 samples, 0.13%)qsl::engine::OrderBook::fill_front_order(std::__cxx11::list<qsl::engine::Order, std::pmr::polymorphic_allocator<qsl::engine::Order> >&, long, qsl::engine::OrderBook::MatchContext&) (1946 samples, 7.45%)qsl::engi..__memcpy_generic (73 samples, 0.28%)_int_free_chunk (32 samples, 0.12%)_int_free_merge_chunk (26 samples, 0.10%)_int_free_create_chunk (18 samples, 0.07%)unlink_chunk.isra.0 (3 samples, 0.01%)cfree@GLIBC_2.17 (328 samples, 1.26%)free@plt (52 samples, 0.20%)memcpy@plt (13 samples, 0.05%)operator delete(void*, std::align_val_t)@plt (28 samples, 0.11%)operator delete(void*, unsigned long, std::align_val_t)@plt (58 samples, 0.22%)operator new(unsigned long) (333 samples, 1.28%)__aarch64_ldadd8_relax (72 samples, 0.28%)__libc_malloc2 (17 samples, 0.07%)_int_malloc (17 samples, 0.07%)malloc (160 samples, 0.61%)malloc@plt (31 samples, 0.12%)std::_Hashtable<unsigned long, std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, std::pmr::polymorphic_allocator<std::pair<unsigned long const, qsl::engine::OrderBook::Locator> >, std::__detail::_Select1st, std::equal_to<unsigned long>, std::hash<unsigned long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_erase(unsigned long, std::__detail::_Hash_node_base*, std::__detail::_Hash_node<std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, false>*) (496 samples, 1.90%)_int_free_chunk (10 samples, 0.04%)_int_free_merge_chunk (7 samples, 0.03%)cfree@GLIBC_2.17 (113 samples, 0.43%)free@plt (30 samples, 0.11%)operator delete(void*, std::align_val_t)@plt (27 samples, 0.10%)operator delete(void*, unsigned long, std::align_val_t)@plt (41 samples, 0.16%)std::pmr::(anonymous namespace)::newdel_res_t::do_deallocate(void*, unsigned long, unsigned long) (9 samples, 0.03%)std::__detail::_List_node_base::_M_unhook() (23 samples, 0.09%)std::__detail::_List_node_base::_M_unhook()@plt (9 samples, 0.03%)std::pmr::(anonymous namespace)::newdel_res_t::do_deallocate(void*, unsigned long, unsigned long) (9 samples, 0.03%)std::_Hashtable<unsigned long, std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, std::pmr::polymorphic_allocator<std::pair<unsigned long const, qsl::engine::OrderBook::Locator> >, std::__detail::_Select1st, std::equal_to<unsigned long>, std::hash<unsigned long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_erase(unsigned long, std::__detail::_Hash_node_base*, std::__detail::_Hash_node<std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, false>*) (538 samples, 2.06%)_int_free_chunk (12 samples, 0.05%)_int_free_maybe_trim (4 samples, 0.02%)_int_free_merge_chunk (7 samples, 0.03%)_int_free_create_chunk (4 samples, 0.02%)cfree@GLIBC_2.17 (109 samples, 0.42%)free@plt (33 samples, 0.13%)operator delete(void*, std::align_val_t)@plt (33 samples, 0.13%)operator delete(void*, unsigned long, std::align_val_t)@plt (57 samples, 0.22%)std::pmr::(anonymous namespace)::newdel_res_t::do_deallocate(void*, unsigned long, unsigned long) (24 samples, 0.09%)std::_Rb_tree_rebalance_for_erase(std::_Rb_tree_node_base*, std::_Rb_tree_node_base&) (731 samples, 2.80%)s..std::_Rb_tree_rebalance_for_erase(std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)@plt (14 samples, 0.05%)std::__detail::_List_node_base::_M_unhook() (34 samples, 0.13%)std::__detail::_List_node_base::_M_unhook()@plt (10 samples, 0.04%)std::pmr::(anonymous namespace)::newdel_res_t::do_deallocate(void*, unsigned long, unsigned long) (31 samples, 0.12%)qsl::engine::OrderBook::rest(unsigned long, qsl::core::Side, long, unsigned int) (8759 samples, 33.54%)qsl::engine::OrderBook::rest(unsigned long, qsl::core..__posix_memalign (4 samples, 0.02%)operator new(unsigned long, std::align_val_t) (940 samples, 3.60%)ope..__posix_memalign (635 samples, 2.43%)_..__libc_malloc2 (84 samples, 0.32%)_int_malloc (78 samples, 0.30%)unlink_chunk.isra.0 (3 samples, 0.01%)_mid_memalign (93 samples, 0.36%)malloc (303 samples, 1.16%)_mid_memalign (70 samples, 0.27%)posix_memalign@plt (36 samples, 0.14%)operator new(unsigned long, std::align_val_t)@plt (56 samples, 0.21%)qsl::engine::OrderBook::level_for[abi:cxx11](qsl::core::Side, long) (5604 samples, 21.46%)qsl::engine::OrderBook::level_for..cfree@GLIBC_2.17 (52 samples, 0.20%)std::pair<std::_Rb_tree_iterator<std::pair<long const, std::__cxx11::list<qsl::engine::Order, std::pmr::polymorphic_allocator<qsl::engine::Order> > > >, bool> std::_Rb_tree<long, std::pair<long const, std::__cxx11::list<qsl::engine::Order, std::pmr::polymorphic_allocator<qsl::engine::Order> > >, std::_Select1st<std::pair<long const, std::__cxx11::list<qsl::engine::Order, std::pmr::polymorphic_allocator<qsl::engine::Order> > > >, std::greater<long>, std::pmr::polymorphic_allocator<std::pair<long const, std::__cxx11::list<qsl::engine::Order, std::pmr::polymorphic_allocator<qsl::engine::Order> > > > >::_M_emplace_unique<long&, std::__cxx11::list<qsl::engine::Order, std::pmr::polymorphic_allocator<qsl::engine::Order> > >(long&, std::__cxx11::list<qsl::engine::Order, std::pmr::polymorphic_allocator<qsl::engine::Order> >&&) (2622 samples, 10.04%)std::pair<std:..__posix_memalign (5 samples, 0.02%)cfree@GLIBC_2.17 (60 samples, 0.23%)free@plt (25 samples, 0.10%)operator delete(void*, std::align_val_t)@plt (18 samples, 0.07%)operator delete(void*, unsigned long, std::align_val_t)@plt (27 samples, 0.10%)operator new(unsigned long, std::align_val_t) (375 samples, 1.44%)__posix_memalign (235 samples, 0.90%)_mid_memalign (52 samples, 0.20%)malloc (112 samples, 0.43%)_mid_memalign (21 samples, 0.08%)posix_memalign@plt (33 samples, 0.13%)operator new(unsigned long, std::align_val_t)@plt (58 samples, 0.22%)std::_Rb_tree_decrement(std::_Rb_tree_node_base*) (153 samples, 0.59%)std::_Rb_tree_decrement(std::_Rb_tree_node_base*)@plt (15 samples, 0.06%)std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&) (317 samples, 1.21%)std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)@plt (47 samples, 0.18%)std::pmr::(anonymous namespace)::newdel_res_t::do_allocate(unsigned long, unsigned long) (3 samples, 0.01%)std::pmr::(anonymous namespace)::newdel_res_t::do_deallocate(void*, unsigned long, unsigned long) (10 samples, 0.04%)std::pair<std::_Rb_tree_iterator<std::pair<long const, std::__cxx11::list<qsl::engine::Order, std::pmr::polymorphic_allocator<qsl::engine::Order> > > >, bool> std::_Rb_tree<long, std::pair<long const, std::__cxx11::list<qsl::engine::Order, std::pmr::polymorphic_allocator<qsl::engine::Order> > >, std::_Select1st<std::pair<long const, std::__cxx11::list<qsl::engine::Order, std::pmr::polymorphic_allocator<qsl::engine::Order> > > >, std::less<long>, std::pmr::polymorphic_allocator<std::pair<long const, std::__cxx11::list<qsl::engine::Order, std::pmr::polymorphic_allocator<qsl::engine::Order> > > > >::_M_emplace_unique<long&, std::__cxx11::list<qsl::engine::Order, std::pmr::polymorphic_allocator<qsl::engine::Order> > >(long&, std::__cxx11::list<qsl::engine::Order, std::pmr::polymorphic_allocator<qsl::engine::Order> >&&) (2692 samples, 10.31%)std::pair<std:..cfree@GLIBC_2.17 (54 samples, 0.21%)free@plt (20 samples, 0.08%)operator delete(void*, std::align_val_t)@plt (23 samples, 0.09%)operator delete(void*, unsigned long, std::align_val_t)@plt (23 samples, 0.09%)operator new(unsigned long, std::align_val_t) (414 samples, 1.59%)__posix_memalign (275 samples, 1.05%)_mid_memalign (54 samples, 0.21%)malloc (133 samples, 0.51%)_mid_memalign (26 samples, 0.10%)posix_memalign@plt (32 samples, 0.12%)operator new(unsigned long, std::align_val_t)@plt (65 samples, 0.25%)std::_Rb_tree_decrement(std::_Rb_tree_node_base*) (168 samples, 0.64%)std::_Rb_tree_decrement(std::_Rb_tree_node_base*)@plt (11 samples, 0.04%)std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&) (311 samples, 1.19%)std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)@plt (33 samples, 0.13%)std::pmr::(anonymous namespace)::newdel_res_t::do_allocate(unsigned long, unsigned long) (5 samples, 0.02%)std::pmr::(anonymous namespace)::newdel_res_t::do_deallocate(void*, unsigned long, unsigned long) (6 samples, 0.02%)std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*) (55 samples, 0.21%)std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*)@plt (47 samples, 0.18%)std::__detail::_Map_base<unsigned long, std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, std::pmr::polymorphic_allocator<std::pair<unsigned long const, qsl::engine::OrderBook::Locator> >, std::__detail::_Select1st, std::equal_to<unsigned long>, std::hash<unsigned long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true>, true>::operator[](unsigned long const&) (1859 samples, 7.12%)std::__de..__posix_memalign (20 samples, 0.08%)operator new(unsigned long, std::align_val_t) (807 samples, 3.09%)op..__posix_memalign (530 samples, 2.03%)__libc_malloc2 (26 samples, 0.10%)_int_malloc (23 samples, 0.09%)_mid_memalign (89 samples, 0.34%)malloc (264 samples, 1.01%)_mid_memalign (40 samples, 0.15%)posix_memalign@plt (60 samples, 0.23%)operator new(unsigned long, std::align_val_t)@plt (154 samples, 0.59%)std::_Hashtable<unsigned long, std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, std::pmr::polymorphic_allocator<std::pair<unsigned long const, qsl::engine::OrderBook::Locator> >, std::__detail::_Select1st, std::equal_to<unsigned long>, std::hash<unsigned long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_insert_unique_node(unsigned long, unsigned long, std::__detail::_Hash_node<std::pair<unsigned long const, qsl::engine::OrderBook::Locator>, false>*, unsigned long) (324 samples, 1.24%)std::__detail::_Prime_rehash_policy::_M_need_rehash(unsigned long, unsigned long, unsigned long) const (51 samples, 0.20%)std::__detail::_Prime_rehash_policy::_M_need_rehash(unsigned long, unsigned long, unsigned long) const@plt (60 samples, 0.23%)std::pmr::(anonymous namespace)::newdel_res_t::do_allocate(unsigned long, unsigned long) (28 samples, 0.11%)qsl::engine::OrderBook::can_store_limit(qsl::core::Side, long, unsigned int, qsl::core::TimeInForce) const (186 samples, 0.71%)qsl::engine::OrderBook::contains(unsigned long) const (928 samples, 3.55%)qsl.. diff --git a/docs/performance/perf-stat.txt b/docs/performance/perf-stat.txt new file mode 100644 index 0000000..660a66d --- /dev/null +++ b/docs/performance/perf-stat.txt @@ -0,0 +1,87 @@ +QSL matching-engine hot path — perf stat (hardware counters), before vs after +============================================================================== + +Optimizations under test (order-book insertion + matching hot path): + - #138 OrderBook::level_for: std::map::emplace -> try_emplace + - #145 OrderBook ctor: index_ unordered_map max_load_factor 1.0 -> 0.25 + +Workload / benchmark command: + qsl-perfeval 60000000 + (steady-state deep book, baseline storage; each "order" = one new_limit that may + match resting liquidity and rest its remainder; the book is held ~512 deep by + cancelling the oldest order each cycle. 60,000,000 orders.) + +Build: + Compiler: GCC (c++) 16.1.1 20260515 (Red Hat 16.1.1-2) + Flags: Release (-O3 -DNDEBUG) + -fno-omit-frame-pointer -g (CMake "flamegraph" preset) + Binary: build/flamegraph/qsl-perfeval + +Hardware / OS: + CPU: Apple M2 (aarch64); scheduled on the Avalanche performance cores + (MIDR CPU part 0x032). Heterogeneous SoC: perf opens each event against + both apple_avalanche_pmu (P-core) and apple_blizzard_pmu (E-core); the + single-threaded workload runs on the P-cores, so the Avalanche rows carry + the real counts and the Blizzard rows read . + Kernel: Linux 6.19.14-400.asahi.fc44.aarch64+16k (Fedora Asahi Remix, bare metal) + Governor: schedutil (/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor) + perf: perf version 6.19.14-400.asahi.fc44.aarch64 + paranoid: 2 + +Counter availability (issue #90): + The Apple Silicon PMU exposed by the Asahi driver implements cycles, instructions, + branches, and branch-misses, but NOT the generic cache-references / cache-misses + events. Cache-miss rate is therefore UNMEASURABLE on this host and is reported as + "unavailable" — never estimated. (A PMU microarchitecture that exposes cache events, + e.g. x86_64 or an ARM server core, is needed to close that gap.) + +------------------------------------------------------------------------------ +BEFORE (std::map::emplace + default index load-factor 1.0) +------------------------------------------------------------------------------ + perf stat -e cycles,instructions,branches,branch-misses -- qsl-perfeval 60000000 + + 20,894,373,501 apple_avalanche_pmu/cycles/u + apple_blizzard_pmu/cycles/u + 74,351,477,053 apple_avalanche_pmu/instructions/u + apple_blizzard_pmu/instructions/u + 14,635,813,879 apple_avalanche_pmu/branches/u + apple_blizzard_pmu/branches/u + 295,293,764 apple_avalanche_pmu/branch-misses/u + apple_blizzard_pmu/branch-misses/u + + Derived (per 60,000,000 orders): + cycles/order 348.2 + instructions/order 1239.2 + IPC 3.56 + branches/order 243.9 + branch-miss rate 2.018 % (295,293,764 / 14,635,813,879) + cache-miss rate unavailable (PMU does not expose cache counters; #90) + +------------------------------------------------------------------------------ +AFTER (try_emplace + index load-factor 0.25) +------------------------------------------------------------------------------ + perf stat -e cycles,instructions,branches,branch-misses -- qsl-perfeval 60000000 + + 17,303,430,508 apple_avalanche_pmu/cycles/u (99.50%) + apple_blizzard_pmu/cycles/u + 68,572,470,652 apple_avalanche_pmu/instructions/u (99.50%) + apple_blizzard_pmu/instructions/u + 13,741,483,668 apple_avalanche_pmu/branches/u (99.50%) + apple_blizzard_pmu/branches/u + 249,062,345 apple_avalanche_pmu/branch-misses/u (99.50%) + apple_blizzard_pmu/branch-misses/u + + Derived (per 60,000,000 orders): + cycles/order 288.4 (-17.2 % vs before) + instructions/order 1142.9 ( -7.8 %) + IPC 3.96 (+11.4 %) + branches/order 229.0 ( -6.1 %) + branch-miss rate 1.813 % (-0.205 pp; -10.2 % relative) + cache-miss rate unavailable (#90) + +Reading: the ~+25% throughput is explained by ~-17% cycles/order. That cycle drop +is part fewer instructions/order (-7.8%: shorter index probe chains + no throwaway +per-insert pmr::list construction) and part higher IPC (+11.4%: shorter probe chains +stall the pipeline/memory system less). The branch-miss rate also falls, consistent +with fewer mispredicted bucket-traversal loop-back branches. Allocations/order are +UNCHANGED (1.106 -> 1.106) — the win is instruction/cycle efficiency, not allocation +count (libstdc++ std::map::emplace checks the key before allocating a node). diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 9c862f5..2008000 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -114,6 +114,11 @@ add_test( "EXPECT_OUTPUT=usage:" -P "${CMAKE_CURRENT_LIST_DIR}/cmake/expect_command_failure.cmake") +# Performance-evidence harness smoke test (PERFORMANCE.md): a tiny run must complete and report. +add_test(NAME qsl_perfeval_smoke COMMAND qsl-perfeval 5000 --latency) +set_tests_properties(qsl_perfeval_smoke PROPERTIES PASS_REGULAR_EXPRESSION + "perfeval: latency_ns") + # Shell unit tests for the shared artifact-publish helper (MAC sanitization + # trailing-whitespace/blank-line trimming). Portable (sed/awk/mktemp); the script # locates the repo via BASH_SOURCE, so it runs correctly by absolute path.