Benchmarks¶
Every number here was measured on one machine: an AWS c7i.xlarge (Intel Xeon Platinum 8488C, 2 cores and 4 threads), Ubuntu 24.04, Linux 7.0, gcc 13.3. Every benchmark, with its spread and throughput: bench/full-results.md.
Hot path¶
Time per operation, preset release-native, pinned to one core.
| Stage | Operation | p50 | p99 | p99.9 | Benchmark |
|---|---|---|---|---|---|
| Market data | Binance book ticker, JSON decode | 116.6 ns | BM_Json_BinanceBookTicker |
||
| Nasdaq ITCH message into the L2 book | 81.9 ns | 90.1 ns | BM_ItchL2Bridge_Message |
||
| L2 book, apply a 20-level delta | 353.3 ns | BM_L2_ApplyDelta/20 |
|||
| Engine | Tick to order, simulator | 159.0 ns | 199.0 ns | 319.0 ns | BM_TickToOrder_Sim |
Strategy: two quotes (basic_mm) |
11.2 ns | BM_BasicMM_ComputeQuotes |
|||
| Quote diff against resting orders | 49.4 ns | BM_QuoteManager_Reconcile_6Orders1Change |
|||
| Pre-trade risk check | 6.7 ns | BM_Risk_CheckNew_Pass |
|||
| OMS: submit an order | 30.9 ns | BM_Oms_Submit |
|||
| Journal: record a 128-byte event | 5.7 ns | BM_Journal_Record/128 |
|||
| Orders | Binance order.place, HMAC-signed |
714.6 ns | BM_Encode_BinanceOrderPlace |
||
Binance order.place, Ed25519 session |
268.2 ns | BM_Encode_BinanceOrderPlace_Session |
|||
| OUCH 5.0 Enter Order | 17.4 ns | 19.5 ns | BM_Ouch50_EncodeNewIds |
End to end over veth¶
fastmm-sim-itch and fastmm-live in two network namespaces joined by a veth pair, basic_mm on configs/nasdaq-itch-sim.toml, preset release, busy polling, simulator on CPU 0, engine on CPU 1 and network thread on CPU 3, three runs of 30 s on 2026-09-29. Each cell is the range over the runs; the engine's own intervals are in bench/README.md.
| Interval | p50 | p99 | p99.9 |
|---|---|---|---|
| Wire to wire | 11.3 to 11.8 µs | 19.5 to 23.6 µs | 21.3 to 33.2 µs |
| Kernel receive to T0 | 2.3 µs | 5.4 to 5.6 µs | 14.8 to 32.8 µs |
| T0 to T5: engine, tick to order | 2.6 to 2.7 µs | 12.8 to 14.3 µs | 14.4 to 18.3 µs |
What each number times¶
Tick to order (BM_TickToOrder_Sim) starts when a book delta is pushed into the engine's feed and ends when the last order it caused is serialised into the transport. In between: the L2 book, the strategy hook, the quote diff, the pre-trade risk checks, the OMS and the encoding of one cancel per side. In the simulator the transport copies each message into a 192-byte scheduler slot, the stand-in for the live engine's outbound ring. The venue, acks and the untimed settle loop between ticks are outside it. The simulator's outbound SHA-256, a determinism check the live engine does not compute, is off; BM_TickToOrder_SimHash turns it on.
Wire to wire is measured by the simulator with its own TSC: from right before the sendmmsg that sent an ITCH datagram to right after the read that returned the order it caused. Each order carries the MoldUDP64 sequence number that triggered it in its ClOrdID, so the simulator matches them. fastmm-live measures the other rows: the kernel receive timestamp to T0 on the network thread, T0 to T5 on the engine (Status file), and T0 to the return of the write that carried the order. On veth that write runs the simulator's TCP receive path inside the system call, which is most of the gap between T5 and wire to wire.
Method¶
scripts/bench.sh --preset release-native --cpu 1runs each benchmark binary twice. The first pass is pinned to one core (tasksetandsched_setaffinity) and skips the benchmarks that declare they need more cores (FASTMM_BENCH_NEEDS_CORES,bench/bench_pin.hpp); the second runs those unpinned. A benchmark that finds fewer cores in its affinity mask than it declared fails instead of reporting a number.- Five repetitions per benchmark, reported raw;
--rounds Nrepeats the suite andtools/bench_table.pypools every repetition of every round. - Where a benchmark times each operation with
rdtscinto aLogLinearHistogram(tick to order, the reactor, UDP, the ITCH and OUCH codecs), the tables give its p50 and p99, and p99.9 for tick to order. The buckets are about 2 % wide. Elsewhere Google Benchmark reports one mean per repetition, and the tables give the median of those means. - Tick to order uses manual timing (
UseManualTime): the reported time is therdtscinterval around the tick. Between timed ticks the rig delivers acks until both quotes rest at the venue, so each timed tick sends orders; the benchmark fails when fewer than half do (order_events_pct). - The codec benchmarks time 64 operations between two
rdtscreadings and report the median per-operation time; the*_NewIdsvariants change the ClOrdID on every call so the compiler cannot hoist the work. - Loops that measure latency chain each result into the next iteration's input (
BM_Risk_CheckNew_Pass,BM_Fixed_Mul,BM_Fixed_RoundToTick,BM_L2_UpdateNearTop).BM_Risk_CheckNew_Killed,BM_Fixed_FromDecimal,BM_Fixed_ToDecimalandBM_L2_InsertEraseTopmeasure throughput and say so in their source. FASTMM_ALIGN_CODE(default on, gcc) aligns functions to 64 bytes and loops and jump targets to 32 bytes on the FastMM targets, so an unrelated edit does not move the tick by changing code layout. Two processes of one binary still differ by 2 to 5 %: compare a change over several interleaved processes per side.- The instance ran nothing else, without
isolcpusor a fixed CPU frequency.bench/full-results.mdgives each row's range over all repetitions next to its median. bench/ci_budget.tomlholds time budgets and counter floors;python3 tools/check_budgets.py bench/results/latestexits 1 when a benchmark's fastest repetition exceeds its budget by more than 10 %, reported an error, or fell below a counter floor. CI runs a subset on a hosted runner with--slack 2.tools/bench_compare.pydiffs two result directories.scripts/build-pgo.sh [--bolt]buildsrelease-nativewith PGO (and BOLT) trained on the hot-path benchmarks, a backtest andscripts/bench-e2e.sh. Shipped builds do not use it.
Across two hosts: Two-host benchmark.