Python strategy API¶
Strategies written in Python run inside the C++ engine. They use the same engine, risk checks, OMS, quote manager, journal and outbound hash as C++ strategies (ADR-0012, section 7). Backtests, data sources and results: Python.
Methods marked @fastmm.hot are compiled with Numba instead and follow Hot hooks.
Scope: a class with plain hooks only (no @fastmm.hot methods) runs in in-process backtests; it does not replay, run in fastmm.sweep or trade live. A class with hot hooks also replays (Replay) and trades live (Live sessions); What runs where compares them.
A strategy¶
import fastmm
from fastmm import BUY, SELL, Param, Strategy
class SkewMM(Strategy):
half_spread_bps = Param(0.01, min=0.0, max=10_000.0, doc="half spread around mid, bps")
quote_qty = Param(0.002, min=0.0, doc="size per level, base units")
def on_start(self, ctx):
ctx.every(100_000_000) # on_timer every 100 ms
def on_book(self, ctx, inst, book):
if not book.valid:
return ctx.pull_quotes(inst)
half = book.mid * self.half_spread_bps * 1e-4
ctx.set_quotes(inst, [(book.mid - half, self.quote_qty)],
[(book.mid + half, self.quote_qty)])
def on_fill(self, ctx, fill):
print(fill.side, fill.price, ctx.position(fill.instrument).qty)
cfg = fastmm.BacktestConfig.from_toml("configs/backtest-example.toml")
cfg.clear_params()
r = fastmm.run_backtest(cfg, data="synthetic", strategy=SkewMM, params={"quote_qty": 0.001})
r.strategy, r.outbound_sha256, r.stats()["net_pnl"]
fastmm.run_backtest(config, data=None, strategy=None, params=None):
strategy= |
runs |
|---|---|
None |
config.strategy (a registered C++ strategy) |
"basic_mm" |
the registered C++ strategy |
a Strategy subclass |
a fresh instance of it |
a Strategy instance |
that instance, once (read its attributes afterwards); a second run raises ValueError |
params= is applied on top of config.params for every kind of strategy. The TOML example configures basic_mm, so call cfg.clear_params() before running a strategy with other parameters. The result's strategy is py:<QualName>, which is also written to the journal header (truncated to 31 characters) when config.journal_out is set.
Hooks¶
Hook names and argument order match the C++ hooks. Define any subset; a hook the class does not define is never looked up or called. Strategy.hooks() lists the defined hooks and raises TypeError for a wrong signature (fastmm: X.on_trade has the wrong signature; expected on_trade(self, ctx, inst, trade)); likely misspellings such as on_fills warn (silence with fastmm_allow_near_miss_names = True).
| Hook | Called |
|---|---|
on_start(self, ctx) |
once before the first event |
on_stop(self, ctx) |
once after the last event |
on_book(self, ctx, inst, book) |
every book update of an instrument in the table |
on_book_ticker(self, ctx, inst, msg) |
every best bid and ask update |
on_trade(self, ctx, inst, trade) |
every public trade |
on_option_ticker(self, ctx, inst, msg) |
every option mark and greeks update |
on_fill(self, ctx, fill) |
every execution, after position and fees are updated and before the on_order_update of the same execution |
on_order_update(self, ctx, update) |
every change of one of our orders |
on_timer(self, ctx, timer_id, tag) |
a timer from ctx.every or ctx.once fired |
on_connection(self, ctx, msg) |
a venue connection state change (not produced by the simulator) |
on_quoting(self, ctx, enabled) |
ctx.quoting_enabled changed (kill switch, operator pull, reconciliation) |
on_balance(self, ctx, msg) |
a venue reported one asset of the account; ctx.balance already holds it |
on_perp_state(self, ctx, inst, msg) |
a venue's mark, index or funding of a derivative arrived; ctx.mark and ctx.funding already hold it |
on_risk_reject(self, ctx, reject) |
the engine's own risk check refused a send, replace or a quote; reject is a RiskReject (inst, side, reason, replace, order_id, tag, post_only, price, qty, time_ns, a copy), delivered after the hook that asked returns (Strategy API) |
inst is an Instrument; book, msg, trade, fill and update are views (below).
A class may also define state(self) -> bytes | str and restore(self, data: bytes) -> bool. With BacktestConfig.state_file (or [strategy] state_file), restore gets the file's bytes once, after on_start, and state is written to the file every state_interval_s and at the end (Strategy API).
Parameters¶
Param(default, min=None, max=None, doc="") declares a typed parameter: the type comes from the default (bool, int or float; schema types bool, int, double). Read it as an attribute (self.quote_qty). Values from config.params (strings) and params= (Python scalars) are parsed like FASTMM_PARAM values in C++, and errors have the same text with the strategy name first:
py:SkewMM: unknown parameter 'levles'
py:SkewMM: parameter 'levels': value 9 outside [1, 8]
py:SkewMM: parameter 'levels': cannot parse '2.5' as int (a whole number)
An unbounded side prints as -inf or inf. Override validate(self) -> Optional[str] for cross-field checks; it runs after all keys are applied, and on any error the previous values stay. Strategy.schema() returns [{"name", "type", "default", "min", "max", "doc"}, ...] like fastmm.strategies().
Context¶
ctx mirrors the C++ StrategyContext. It is usable only inside a hook, on the thread running the backtest; anywhere else it raises RuntimeError. Instrument arguments accept an Instrument or its integer id; side is fastmm.BUY (0) or fastmm.SELL (1).
| Group | API |
|---|---|
| Time, reference data | now_ns, instruments (tuple, index == id), instrument(inst), contains(inst) |
| Market data | book(inst) -> BookView |
| Execution view | own_qty(inst, side, price, at_ns=None), own_qty_raw(inst, side, price_raw, at_ns=None), best_ex_self(inst, side) -> (price, qty), best_ex_self_raw(inst, side), queue_ahead(order_id) -> float or None, queue_ahead_raw(order_id) -> int or None (Execution view) |
| Portfolio | position(inst) -> PositionView, portfolio() -> Portfolio (realized, unrealized, fees, net) |
| Quoting | set_quotes(inst, bids, asks) -> bool, set_quotes_raw(inst, bids, asks) -> bool, pull_quotes(inst), pull_all_quotes(), working_quote(inst, side, level=0) -> Order or None |
| Direct orders | send(inst, side, price, qty, *, post_only=False, reduce_only=False, ioc=False, tag=0) -> int, send_raw(...), cancel(order_id) -> bool, replace(order_id, price, qty), replace_raw(...), order(order_id) -> Order or None, open_qty(inst, side), open_qty_raw(inst, side) |
| Timers | every(period_ns, tag=0) -> int, once(delay_ns, tag=0) -> int, cancel_timer(timer_id) -> bool |
| Control | quoting_enabled, killed, request_stop() (the run ends after the current event) |
| Venue state | fees(inst) -> Fees (maker_bps, taker_bps, maker_cbps, taker_cbps), risk_headroom(inst) -> RiskHeadroom, venue_health(venue=0) -> VenueHealth, order_budget(venue=0) -> OrderBudget (Strategy API) |
| Balances | balance(venue, asset) -> Balance, margin(venue=0) -> Margin, balance_room(inst, side, price) -> float or None, balance_room_raw(inst, side, price_raw) -> int or None, balances_live() (Balances) |
| Perpetuals | mark(inst), index(inst) -> RefPrice, funding(inst) -> Funding (Perpetuals) |
| Randomness | random() in [0, 1), randint(lo, hi) inclusive, from the engine's seeded RNG |
set_quotes takes sequences of (price, qty) pairs, level 0 first, at most 8 per side (None is an empty side). It returns False, and the quotes are ignored, while quoting is disabled. More than 8 levels, a non-finite price or quantity, or a negative quantity raise ValueError before the engine is touched; a non-positive price or a zero quantity drops the level, as DesiredQuotes::bid/ask do in C++.
RiskHeadroom has order_tokens, open_orders (int), max_order_qty, max_order_notional, buy_qty, sell_qty, underlying_buy_qty, underlying_sell_qty, gross_notional, net_buy_notional, net_sell_notional, exposure_buy_notional, exposure_sell_notional, loss_budget, balance_buy_qty, balance_sell_qty (float, each with a _raw int); a limit that is off is None. VenueHealth has feed_lag_ns, feed_lag_base_ns, feed_lag_excess_ns, ack_rtt_ns, ack_rtt_smoothed_ns, md_updated_ns, ack_updated_ns, md_samples, ack_samples, gate_engagements and gated. OrderBudget has local_tokens (the [risk] orders_per_sec bucket; None with the limit off), orders_10s, orders_1m, orders_1d and weight (each (used, limit) or None where the venue declares no such window), venue_paused, venue_known (False in a backtest) and orders_remaining (counted up to where the connector stops sending, 90 % of each limit). Hot hooks (@fastmm.hot) do not see these.
Balance has free, locked, total (the venue's last report moved by this engine's orders and fills since), equity, maintenance (as reported; each float with a _raw int), as_of_ns and known (False until the venue reports the asset). Margin has available, initial, maintenance, equity, wallet (float and _raw), asset, as_of_ns, account (the account-wide margin) and known. balance_room is None while the venue has not reported the balance the side draws on.
RefPrice has price (float and price_raw), at_ns, stale and usable. Funding has rate (per interval; positive: longs pay), interval_ns, next_ns (0: continuous), at_ns, stale, usable and over(seconds), the rate over a holding time.
send and replace raise fastmm.OrderRejected with .reason, the RejectReason name ("InvalidTag" for a tag in the quote manager's range, "UnknownOrder", risk reasons such as "MaxPosition"). cancel returns False for an unknown or terminal order.
Views and values¶
Views are reused between hooks. A view is valid only inside the hook that received it, or the hook that called ctx.book() / ctx.position(); reading it later raises fastmm.StaleViewError (a RuntimeError). Copy the numbers you want to keep.
| View | Fields |
|---|---|
BookView |
instrument, valid, mid, spread, best_bid and best_ask as (price, qty), level(side, i), depth(side), last_update_ns, seq, microprice(), imbalance(levels=1) |
PositionView |
instrument, qty, avg_price, realized, unrealized, fees, net_pnl, fills |
TradeView |
instrument, price, qty, aggressor (side), trade_id, exch_ts_ns, recv_ts_ns |
BookTickerView |
instrument, bid_price, bid_qty, ask_price, ask_qty, exch_ts_ns, recv_ts_ns |
OptionTickerView |
instrument, mark_price, underlying_price, index_price, mark_iv, bid_iv, ask_iv, delta, gamma, vega, theta, rho, interest_rate, exch_ts_ns, recv_ts_ns |
BalanceView |
venue, asset, free, locked, total, equity, maintenance, snapshot, snapshot_end, account, exch_ts_ns, recv_ts_ns |
PerpStateView |
instrument, venue, has_mark, has_index, has_funding, has_open_interest, mark_price, index_price, funding_rate, funding_interval_ns, next_funding_ns, open_interest, exch_ts_ns, recv_ts_ns |
ConnectionView |
venue, state ("Live", "Disconnected", ...), live, channel, reason_code, recv_ts_ns |
FillView |
instrument, side, price, qty, position_delta, fee, fee_converted, liquidity (fastmm.MAKER, fastmm.TAKER), known, late, order_done, order_id, exch_ts_ns, update (OrderUpdateView or None) |
OrderUpdateView |
order_id, instrument, side, state, prev_state, reject_reason, user_tag, known, changed, terminal, price, qty, filled, leaves, fill_price, fill_qty, sent_ns, venue_ack_ns, local_ack_ns |
Instrument (id, symbol, venue, base, quote, asset_class, tick, lot, min_qty, min_notional, contract_multiplier, round_price(price, side), round_qty(qty)), Order (id, instrument, side, state, price, qty, filled, leaves, user_tag, post_only, reduce_only, created_ns, sent_ns, venue_ack_ns, local_ack_ns; the three times are OrderTimes, 0 when not known) and Portfolio are copies and can be kept. An Instrument compares and hashes like its id and works as a list index.
Numbers¶
Every price, quantity and amount is available two ways: a float (book.mid, fill.price) and the exact raw int64 at a 1e-8 scale with a _raw suffix (book.mid_raw, fill.price_raw), the units of C++ Price::raw and of the result columns. There is no Decimal.
set_quotes,sendandreplacetake floats: prices are converted to 1e-8 (half away from zero) and rounded to the tick passively (bids down, asks up), quantities down to the lot.set_quotes_raw,send_rawandreplace_rawtake ints and use them as given.- To send the same orders as a C++ port, use the raw values and the C++ operators:
Fixed * Ratiotruncates toward zero (Python's//floors; seetdivandmul_ratioinexamples/python/strategies/basic_mm_exact.py).
Errors¶
A hook that raises stores the exception, pulls all quotes and asks the engine to stop; no hook of that strategy is called again (including on_stop). The simulator ends the run after the failing event and run_backtest raises fastmm.StrategyError:
try:
fastmm.run_backtest(cfg, data="synthetic", strategy=MyMM)
except fastmm.StrategyError as e:
e.__cause__ # the original exception, with its traceback
e.result # the partial BacktestResult
e.hook, e.now_ns, e.events
KeyboardInterrupt and SystemExit propagate unchanged, with the partial result as .result. The adapter checks for Ctrl-C (PyErr_CheckSignals(), GIL released briefly) every 1,024 hook calls and every 4,096 engine steps.
Determinism¶
A Python strategy is as deterministic as its own code. The engine clock, event order, RNG and outbound hash are the ones C++ strategies use, so two runs with the same configuration, data and seed give the same outbound_sha256 (also across PYTHONHASHSEED values, which the tests check).
- Use
ctx.now_ns, nevertime.time(). - Use
ctx.random()andctx.randint(), neverrandomornumpy.randomwithout a fixed seed. - No threads.
- No decisions that depend on the iteration order of a
setof strings or onid(). - Float results are reproducible on one platform and Python version.
Performance¶
The GIL is held for the whole run of a Python strategy (C++ strategies release it). Measured with python bench/python/bench_py_strategy.py --seconds 3600 --repeat 5 (1,356,426 market-data events of the synthetic L2 queue market; gcc 13 release module, Python 3.12, AMD Ryzen 7 7800X3D under WSL2):
| Case | Wall time | Events/s end to end |
|---|---|---|
C++ basic_mm |
0.670 s | 2.02 M |
Python BasicMMExact (same orders as C++) |
1.035 s | 1.31 M |
Python SkewMM |
1.230 s | 1.10 M |
| Python, no hooks defined | 0.477 s | 2.84 M |
| Python, an empty hook on every event | 0.530 s | 2.56 M |
| Python, two field reads on every event | 0.821 s | 1.65 M |
The end-to-end numbers include the synthetic market and the simulated venue, which dominate here. Per call: an empty hook adds about 40 ns, reading two or three fields about 250 ns, and BasicMMExact about 3.1 µs per on_book or on_fill call over C++ basic_mm.
Limits¶
fastmm.sweeptakes registered C++ strategy names only;fastmm-livenever links Python.- The simulator produces no connection state changes and the numpy source no option tickers, so
on_connectionandon_option_tickerfire only with data sources that contain them. No backtest source carries balances:on_balancedoes not fire andctx.balancestays unknown. A journal recorded byfastmm-livecarries the venues' marks and funding (on_perp_state); the CSV, Tardis, Binance and numpy sources do not. - One thread: a hook must not start threads that call the context.
Hot hooks¶
A class with @fastmm.hot methods is compiled with Numba in nopython mode, and the engine thread calls the compiled hooks through function pointers with the GIL released. It needs numba 0.61 to 0.67 and the hot extra (Install). Steps: Write hot hooks in Python.
Declarations¶
| Declaration | Meaning |
|---|---|
@fastmm.hot on on_book, on_fill, on_quoting, on_connection or on_params |
an event hook |
@fastmm.hot(every="100ms") on any other name |
a timer hook, at most 16; units ns, us, ms, s, m, h |
fastmm.Param(default, min=, max=, doc=) |
a parameter, typed, parsed and validated as in Parameters |
fastmm.State(default, doc="") |
a per-instrument bool, int64 or float that hooks read and write; it starts at the default and keeps its value between calls |
Every hot hook takes (self, ctx, book) and runs once per instrument:
| Hook | Runs for |
|---|---|
on_book |
the instrument whose book changed |
on_fill |
the fill's instrument, after position and fees are updated |
on_quoting |
every instrument, when quoting is enabled or disabled |
on_connection |
every instrument of the venue whose connection changed |
on_params |
every instrument a parameter update applied to, with the new values in self |
| a timer hook | every instrument, once per period of engine time from the start of the run |
self holds the instrument's parameters and State fields as attributes. A float parameter also has self.<name>_raw, its value as a 1e-8 fixed-point int64 (nearest). The engine copies the parameters into self before every call, so an assignment to a parameter, also through an alias such as t = self, is gone at the next call.
Defining the class raises TypeError when it also defines a fastmm.Strategy hook other than on_start and on_stop as a plain method, a hot hook has another name or signature, a name is both a Param and a State, a name clashes with a float parameter's _raw field or with a ctx method, or a hook assigns self.<parameter> (a check of the source). Without numba it raises ImportError naming the hot extra and Install from source.
ctx¶
The hot ctx has no execution view (own_qty, queue_ahead, order times); a hot strategy that needs them runs as a plain Strategy.
| Field | Meaning |
|---|---|
now_ns, instrument |
engine time (ns) and the instrument id |
tick, lot, min_qty, position |
floats (quote currency for the tick, base units otherwise), each with a _raw field |
quoting_enabled |
1 while quoting is enabled and the instrument's venue is not killed; the engine ignores quotes otherwise |
connected |
in on_connection: 1 when the venue's connection is live |
fill_side, fill_maker, fill_price, fill_qty |
in on_fill: fastmm.BUY (0) or fastmm.SELL (1), 1 for a maker fill, price and quantity with _raw fields; 0 in other hooks |
| Method | Intent |
|---|---|
quote(bid_px, ask_px, qty) |
replace the ladder with one level per side |
bid(px, qty), ask(px, qty) |
append a level; levels after the eighth per side are dropped |
quote_raw, bid_raw, ask_raw |
the same with int raw values (1e-8 scale) |
clear() |
an empty ladder: the working quotes are cancelled |
pull() |
pull the instrument's quotes |
uncross() |
move a level-0 ask at or below the level-0 bid to one tick above it (DesiredQuotes::uncross) |
keep_passive() |
shift each side so level 0 is one tick inside the book's touch (keep_passive), after uncross |
fail(code) |
stop the strategy with an int code |
When the hook returns, the engine makes at most one call for the instrument: set_quotes after quote, bid, ask or clear, or pull_quotes after pull, whichever came last. A float price or quantity is rounded to the nearest 1e-8, then prices to the tick (bids down, asks up) and quantities down to the lot; a raw level is used as given. A level with a non-positive price or a zero quantity is dropped.
book¶
| Field | Meaning |
|---|---|
valid, ts_ns |
1 when the book is valid (BookView.valid); last update in engine time, 0 before the first |
mid, best_bid, best_ask, best_bid_qty, best_ask_qty |
floats, each with a _raw field |
n_bids, n_asks |
levels present in the arrays, at most 10 per side |
bid_px[i], bid_qty[i], ask_px[i], ask_qty[i] |
level i, 0 best, each with a _raw array; entries from n_bids or n_asks on are 0 |
fastmm.fx¶
fastmm.fx has the C++ fixed-point operators for raw values; each works in plain Python and in hooks. Constants: SCALE (100,000,000), RATIO_PER_BP (10,000), BUY, SELL.
| Function | Result |
|---|---|
tdiv(a, b) |
integer division truncating toward zero, as in C++; ZeroDivisionError when b is 0 |
mul_ratio(v, r) |
Fixed * Ratio: the 128-bit product divided by 1e8, truncating toward zero |
bps_ratio(bps_raw) |
the Ratio raw value of a bps parameter's _raw field (0.01 bps: 1,000,000 gives 100) |
round_price(p, tick, side), round_qty(q, lot) |
round_to_tick (bids down, asks up) and round_to_lot (down) |
to_raw(x), to_float(raw) |
the nearest raw value (halves away from zero; ValueError when not finite) and back |
What compiles¶
Hooks use Numba's nopython subset: numbers, the fields and methods above, loops, tuples and functions decorated with numba.njit. The code runs with bounds checks (an index outside an array raises IndexError) and Python's error model (a division by zero raises ZeroDivisionError); helpers are compiled with bounds checks when a hook first calls them.
Before a run, FastMM scans the LLVM IR of each hook and of the helpers it calls. A call to anything other than an LLVM intrinsic, a libm function or NRT_MemInfo_call_dtor raises fastmm.HotCompileError (a TypeError) with the hook, the line that allocates or prints when Numba's typed IR shows it, and the symbols:
fastmm: Allocates.on_book is rejected by the IR check: it creates an array at strategy.py:21. Hot hooks may not allocate, print or use Python objects (it calls NRT_MemInfo_alloc_aligned).
This rejects arrays, lists, strings, print and Python objects. Calls through ctypes or cffi function pointers are not detected and are not supported. A hook that Numba cannot compile raises HotCompileError with the hook's name and Numba's message.
Running and errors¶
run_backtest(..., hot_cache=True) compiles the hooks, calls each once on scratch copies of ctx, book and self (nothing is sent and State is unchanged), then runs the backtest with the GIL released.
A hook that raises, calls ctx.fail(code), or sets a float price or quantity that is not finite, beyond 9.2e10 in magnitude, or a negative quantity, stops the strategy: no hook runs again, the kill switch trips with reason StrategyError (quotes pulled, working orders cancelled) and the backtest ends after that event. run_backtest raises fastmm.StrategyError:
| Attribute | Value |
|---|---|
hook |
the hook's name |
status |
1 raised, 2 ctx.fail, 3 bad float level |
fail_code |
the code passed to ctx.fail |
kill_reason |
"StrategyError" |
now_ns, events, result |
engine time, engine events and the partial BacktestResult |
Numba keeps neither the type nor the message of the exception.
Numba cache¶
With hot_cache=True compiled hooks are stored under $FASTMM_CACHE_DIR/numba/ (default $XDG_CACHE_HOME/fastmm or ~/.cache/fastmm) in a directory whose name holds the fastmm version, the hot ABI, a hash of FastMM's compiler sources, the numba and llvmlite versions and the CPU. Numba checks only the source file of each hook: after changing a helper in another file, pass hot_cache=False or delete the directory.
Performance¶
Measured with python bench/python/bench_hot_strategy.py --build build/release (gcc 13 release module, numba 0.67.0, Python 3.12, AMD Ryzen 7 7800X3D under WSL2, one core). Cost of one on_book call without the engine, on the same books, positions and parameters (levels = 2):
| Case | ns per call |
|---|---|
C++ BasicMM::on_book |
31.7 |
BasicMMHot (exact) |
36.8 |
BasicMMHot with bounds checks off |
37.0 |
BasicMMHot with the 10 levels per side copied |
72.0 |
BasicMMHotFloat |
30.4 |
Bounds checks cost less than the run-to-run spread (under 1 ns per call). The engine copies the level arrays only when a hook, or a numba.njit function it calls, names one of them, or calls code the check cannot follow; the copy costs about 35 ns per call.
On the synthetic market (3,600 s, 1,356,426 market-data events) C++ basic_mm runs at 1.93 M events/s and BasicMMHot at 1.90 M events/s with the same orders. Compiling BasicMMHot (5 hooks) takes 1.1 s in a fresh process, 0.5 s from the Numba cache; importing numba takes 0.14 s.
Slow methods¶
A class with hot hooks may also define on_start(self, ctx), on_stop(self, ctx) and methods decorated @fastmm.every(period, timeout="10s") that take (self, ctx). They are plain Python, run one at a time on the slow thread and may use any library. They cannot place orders; they change the hot hooks' parameters by publishing. Steps: Run slow methods beside hot hooks.
| Method | Runs |
|---|---|
on_start |
once, before the engine starts |
@fastmm.every("1s") |
when the session starts, then once per period of session time; units ns, us, ms, s, m, h |
on_stop |
once, after the engine stops |
Defining the class raises TypeError when an @fastmm.every method has another signature or a hook's name, the class has @fastmm.every methods but no hot hooks, it has more than 128 parameters, or a parameter or State field is named publish or inst.
Slow ctx¶
| Member | Result |
|---|---|
now_ns |
session time of the call, ns (simulated in backtests, time.time_ns() live) |
instruments |
instrument symbols by id |
snapshot() |
a Snapshot of the latest state the engine published |
recent(inst) |
Recent(rows, dropped): recent top-of-book changes and trades of one instrument |
fills() |
the fills since the previous call |
publish(inst=None, **values) |
new parameter values; True when the session took the update |
inst is an instrument id or symbol.
Snapshot¶
The engine publishes a snapshot at most once per 10 ms of engine time; a change the interval holds back is published at the end of the interval. A Snapshot has ts_ns (engine time), age_ms (now_ns minus ts_ns in ms; None before the first snapshot), version, quoting_enabled (the engine's quoting flag, false while parameters are stale), killed (the global kill switch) and one InstrumentSnapshot per instrument, indexed by id or symbol (snap["BTCUSDT"]):
| Field | Meaning |
|---|---|
book_valid, book_ts_ns |
the book is valid; its last update in engine time |
bid, bid_qty, ask, ask_qty, mid |
top of book (quote currency, base units) |
position, avg_price, fills |
the position, its average entry price and the fill count |
realized_pnl, unrealized_pnl, fees |
quote currency; unrealized_pnl at the position's last mark |
bid_open_qty, ask_open_qty |
unfilled quantity of open orders, quotes included |
quoting |
quoting is enabled and the instrument's venue is not killed |
param_seq, param_age_ms |
the number of the last update applied to the instrument and its age in ms (None before the first) |
Recent rows¶
recent(inst).rows is a numpy array of the newest rows, oldest first; the engine keeps 4,096 rows per instrument (recent_rows). A row is written when the top of book changes and for every trade:
| Column | Meaning |
|---|---|
ts_ns |
engine time |
kind |
0 the top of book changed, 1 a trade |
bid, bid_qty, ask, ask_qty |
top of book at that time |
mid |
the mean of bid and ask; NaN when a side is empty |
price, qty, side |
the trade and its aggressor (fastmm.BUY or fastmm.SELL); 0 in kind 0 rows |
dropped counts the rows written since the previous recent call for the instrument that left the window before this call.
Fills¶
fills() returns a numpy array with the columns seq (from 1, consecutive over the session), ts_ns, instrument, side, maker, price, qty, fee (quote currency) and position (after the fill). Order updates do not reach slow methods.
The engine writes fills into a ring that the slow tier empties whenever it runs; they wait in Python until fills() takes them. The ring holds fills_capacity fills: by default 4 per order at [risk] orders_per_sec (1,000 when unlimited) over the longer of the shortest period and the longest timeout plus 1 s, at least 4,096, rounded up to a power of two. A full ring stops the session.
Publishing parameters¶
publish names any subset of the parameters. The values apply together at one engine event and the other parameters keep their values; inst=None applies them to every instrument. The hot on_params hook then runs for each instrument the update applied to.
The call checks each name, parses the value and checks its range as fastmm.Param does, then runs validate() on a copy of the instance with the new values, and raises ValueError at the first error; nothing is sent then. An update names at most 32 parameters. An update without values changes nothing and renews max_param_age_ms.
strategy.publish(inst=None, **values) does the same from any Python thread while a session of that instance runs, for a model outside the slow methods such as a thread started in on_start. It returns False before on_start runs and after the session stops. In a backtest its update applies at the simulated time the engine has reached when it next checks for updates, which depends on thread timing.
With max_param_age_ms above 0, quoting is disabled before the first update and while none has applied for that long (Parameter updates). For a class with @fastmm.every methods, a configured value of 0 means 3 times the shortest period, at least 1,000 ms.
In backtests¶
run_backtest calls on_start before the engine starts, each @fastmm.every method at its simulated times after the other events of that time, and on_stop after the run. Pending runs of slow methods do not extend a run past its data.
| Argument | Default | Meaning |
|---|---|---|
slow_delay_ms |
0 | simulated ms from a publish to the event that applies it |
max_param_age_ms |
None |
overrides [strategy] max_param_age_ms; None keeps the configured value or the default above, 0 disables |
fills_capacity |
None |
fills the ring holds |
recent_rows |
4096 | rows per instrument that recent keeps |
BacktestResult.slow_methods maps each @fastmm.every method to calls, p50_ms, p99_ms and max_ms of its wall time. run_backtest issues a RuntimeWarning when a method's median wall time is at least 1 ms and above slow_delay_ms: a live session applies that method's publishes at least that much later. timeout is not enforced in backtests.
A slow method that raises, or a full fills ring, ends the run after that event and run_backtest raises fastmm.StrategyError:
| Attribute | Value |
|---|---|
slow_failure |
"exception" or "fills overflow" |
hook |
the method that raised |
__cause__ |
the exception |
result |
the partial BacktestResult |
Replay¶
fastmm.replay(journal, MyMM, verify=True, config=None, params=None, param_updates=True) replays a journal written by a backtest with config.journal_out or by a live session. The engine consumes the recorded inputs, parameter updates included, with MyMM's hot hooks; slow methods do not run. The parameters start at the recorded values, max_param_age_ms is the recorded one, and the configuration is the one the journal embeds unless config is given. With verify=True every sent message is compared with the recording.
A journal's strategy_meta holds the keys listed under Live sessions, max_param_age_ms and param.<name> for each starting parameter; without param. keys (a journal from an earlier version) the parameters come from the embedded configuration. A difference in class, hot_source_sha256, fastmm or numba, params, or param_updates=False makes the replay a what-if run: what_if is True and what_if_reasons lists the differences.
ReplayResult field |
Meaning |
|---|---|
ok |
the replay sent the recorded messages |
outbound_sha256, recorded_sha256 |
hashes of the replayed and recorded messages |
outbound_messages, recorded_messages, events |
message and inbound event counts |
first_mismatch, expected_message, actual_message |
the first differing message (-1 when none) |
what_if, what_if_reasons |
see above |
Live sessions¶
fastmm.run_live(strategy, config, params=None, *, duration=None, dry_run=False, journal=None, no_journal=False, status=None, no_status=False, log=None, record_raw=None, allow_inline_secrets=False, fills_capacity=None, recent_rows=4096, slow_tier_timeout_ms=10000) runs a class with hot hooks against the venues in config, in this process, and returns the exit code (Exit codes). It needs the live runtime, fastmm-engine-live (Install), and raises ImportError without it. Steps: Run a Python strategy live.
| Argument | Meaning |
|---|---|
strategy |
a fastmm.Strategy subclass with @fastmm.hot methods, or an unused instance |
config |
a fastmm-live configuration (Configuration) |
params |
overrides on top of [strategy.params], which apply only when [strategy] name is py:<Class>, <module>:<Class> or <Class> |
duration |
seconds, or a string such as "60s"; None runs until SIGINT or SIGTERM |
fills_capacity, recent_rows |
as in run_backtest (In backtests) |
slow_tier_timeout_ms |
how long to wait for the slow thread after the session stops |
| the other keywords | the fastmm-live options of the same names (Command lines) |
Before any venue is contacted, run_live applies the parameters, compiles the hooks without the Numba cache and calls each once on scratch data. It then runs the fastmm-live session with the GIL released: the same threads, log, status file, journal, kill switch, on_kill and exit codes.
| Case | Result |
|---|---|
a configuration, parameter or compilation error, a class without hot hooks, or on_start raises |
a message on stderr; returns 3 |
a hook raises, calls ctx.fail or sets a bad float level |
the kill switch trips with StrategyError and stderr names the hook; returns 6 with on_kill = "exit" |
| the slow tier fails (Slow methods live) | the session stops as on SIGTERM; returns 7 |
run_live while a session runs in the process |
RuntimeError, before on_start runs |
run_live in a child forked while a session runs |
RuntimeError; start children with the spawn or forkserver method |
- SIGINT and SIGTERM stop the session. The session replaces the process's handlers while it runs and restores them when it returns, so
KeyboardInterruptis not raised. - Before the engine starts, every thread of the process, including numpy's BLAS threads, is limited to the CPUs not listed in
[engine] cpuandnet_cpus; threads started later inherit that. Without pinned CPUs nothing changes. Thread-count variables such asOPENBLAS_NUM_THREADSact only when set before numpy loads. - The journal's parameter table lists the class's parameters.
inspect_journal(path)["strategy_meta"]returnsclass(module:qualname),hot_source_sha256(the source of the hot hooks and thenumba.njitfunctions they call), thefastmm,numba,llvmliteandpythonversions,max_param_age_msandparam.<name>for each starting parameter (Journal format).fastmm.replayreplays it (Replay). strategy.publishworks fromon_startuntil the session stops; after that it returnsFalse.
Slow methods live¶
run_livemoves the calling thread off the CPUs in[engine] cpuandnet_cpus, so the threads it starts next inherit that, and runson_starton the calling thread before any venue is contacted.on_starthas no timeout.- The slow thread (
fastmm-slow) starts and runs each@fastmm.everymethod at once and then once per period of wall-clock time. Between calls it empties the fills ring every 10 ms. - The session starts. The engine publishes snapshots, recent rows and fills into the slow channel as in backtests. Publishes from
ctx.publishandstrategy.publishshare one ring to the engine; a publish returnsFalsewhen the ring is full. - The control thread checks the slow tier every 50 ms without the GIL. A failure stops the session like SIGTERM: kill switch, cancel-all, exit code 7, or 5 when a cancel-all fails.
- When the session has stopped, the slow thread finishes its current call, runs
on_stopand ends. An exception inon_stopis printed and does not change the exit code. If the thread has not endedslow_tier_timeout_msafter the session stopped,run_liveissues aRuntimeWarningand returns; the thread keeps running as a daemon thread.
| Failure | Log line (ERROR) | stderr |
|---|---|---|
| a slow method raised | fastmm-live: slow tier failed (Exception): a slow method raised |
fastmm: py:<Class>.<method> raised <Type>: <message> and the traceback |
a call ran past its timeout |
fastmm-live: slow tier failed (Timeout): a slow method ran past its timeout |
fastmm: py:<Class>.<method> ran past its timeout (<n> s); the slow tier stopped the session |
| the fills ring was full | fastmm-live: slow tier failed (FillsOverflow): the fills ring was full |
fastmm: py:<Class>: the fills ring (<n> fills) was full; ...; pass a larger fills_capacity |
| the slow thread ended while the session ran | fastmm-live: slow tier failed (ThreadExited): the slow thread ended |
fastmm: py:<Class>: the slow thread ended while the session ran; ... |
The log line is followed by fastmm-live: shutting down (slow tier failed). A method that stops publishing (a stall) disables quoting after max_param_age_ms, which the engine logs as no parameter update for <n> ms (max_param_age_ms): quotes pulled until the next; this comes before the timeout stops the session when max_param_age_ms is below the method's timeout.
python -m fastmm run¶
python -m fastmm run <module>:<Class> --config <file> [options] imports the class, with the current directory first on the module path, and exits with the code run_live returns. The options are the fastmm-live ones: --param key=value (repeatable), --duration, --dry-run, --record-raw, --journal, --no-journal, --status, --no-status, --log and --allow-inline-secrets, and --slow-tier-timeout-ms <ms> (default 10000). When the slow thread has not ended within that time after the session stopped, the process prints fastmm: exiting with code <n> without waiting for the slow thread and exits with os._exit. A bad command line exits with 2 and a class that cannot be imported with 3. Each of OPENBLAS_NUM_THREADS, OMP_NUM_THREADS and MKL_NUM_THREADS that is not set is set to 1, and the interpreter starts again once so that numpy sees them.