Operating a running session¶

A running fastmm-live session listens on a control socket. fastmm-ctl sends it one command and prints the reply:

fastmm-ctl --name sim-local status
fastmm-ctl --name sim-local pull --instrument BTCUSDT
fastmm-ctl --name sim-local flatten --max-slippage-bps 30

Every command except param, status, stop and handoff becomes a message on the engine's control ring, so the journal records it and a replay reproduces the session exactly (Determinism). param is validated against the strategy's schema on the control thread and reaches the engine as a ParamUpdate, which the journal records too, with its --source.

The socket¶

<journal_dir>/<engine name>.ctl, AF_UNIX SOCK_SEQPACKET, mode 0600, created when the session starts and removed when it shuts down. fastmm-live --control <path> moves it; --no-control leaves it out.

Authorisation is the file system: nothing but the socket's mode is checked. Keep journal_dir out of shared directories; run the session and fastmm-ctl as the same user, or give the directory a group and chmod g+x.

One datagram is the request, one datagram is the reply; any SOCK_SEQPACKET client works:

echo status | socat - UNIX-CONNECT:runs/mm.ctl,socktype=5

(socktype=5 is SOCK_SEQPACKET; nc -U speaks stream and datagram sockets only.)

The control thread reads the socket every 50 ms. A command reaches the engine at its next step; the reply means the command is queued, not that the engine has run it. fastmm-ctl status or fastmm-top shows the effect.

The commands¶

Command	What it does
`pull [--instrument SYM \\| --venue NAME]`	Stops quoting and pulls the quotes. Without a scope the whole session stops quoting; with one only that instrument or venue does, and the rest keeps trading. Working orders that are not quotes stay.
`resume [--instrument SYM \\| --venue NAME]`	Quotes again. Without a scope it also clears every scoped pull and stops a running flatten.
`param <name>=<value> ... [--instrument SYM] [--source WHO]`	New strategy parameters. Validated here first: an unknown name, a value out of range or a failed `validate()` is refused and nothing is published (Strategy parameters). The reply `ok param queued seq=N` means the update is on the engine's ring, not that the engine has applied it (below). `--source` records who asked for it (`manual`, `scheduled:drain`, at most 39 bytes) in the journal and the status.
`params [--instrument SYM]`	The control socket's sequence numbers, published and applied, the source of the last update, then the parameters' values, one `name=value` per line (below).
`limits <key>=<value> ...`	New risk limits. The keys are the `[risk]` keys and `underlying.<BASE>.max_net` for a base asset with a `[risk.underlying.<BASE>]` section; the ones you do not name keep the values the session started with.
`flatten [--instrument SYM] [--max-slippage-bps N]`	The engine works the position off itself (below).
`kill`	Trips the global kill switch: quoting off, every quote pulled, every working order cancelled. The position stays.
`unkill`	Clears it and resumes quoting, as `SIGHUP` does, including a latched `max_loss` trip (Kill switch and shutdown).
`stop`	Shuts the session down as `SIGTERM` does: kill switch, cancel-all on every venue, exit.
`handoff`	`stop` for the process waiting on the instance lock (`fastmm-live --standby` or `--takeover`): the same shutdown, then the lock is released last. Refused by a session that holds no lock (Hand over a running session).
`status`	The `fastmm-top` frame plus the limits the session runs with.
`help`	The command list.

--instrument takes a symbol, or <venue>:<symbol> when the same symbol trades on more than one venue. A scoped pull stays until a matching resume: unkill does not clear it.

Replies start with ok or error (fastmm-ctl exit code 0 or 1), except status and help.

fastmm-ctl --name mm param half_spread_bps=99999

error parameter 'half_spread_bps': value 99999 outside [0, 10000]

fastmm-ctl --name mm limits max_position=0.5 orders_per_sec=10

ok limits queued (2 key(s))

New limits take effect on the next pre-trade check, except price_collar_bps and fat_finger_bps, whose bands are recomputed on the next book or trade update of each instrument.

Confirm a parameter change¶

param answers once the update is queued. The engine applies it at its next step and then reports the update's sequence number as applied; --wait <ms> makes fastmm-ctl wait for that, and exit 4 when it does not come in time:

fastmm-ctl --name sim-local --wait 5000 param half_spread_bps=7 --source scheduled:drain

ok param queued seq=1 source=scheduled:drain (every instrument)
ok param applied seq=1

params reads the values back:

fastmm-ctl --name sim-local params

ok published=1 applied=1 pending=0 engine_updates=1 last_seq=1 last_origin=control last_source=scheduled:drain
values=applied
half_spread_bps=7
skew_bps_per_unit=1
quote_qty=0.001
max_inventory=0.01
requote_threshold_ticks=1
pull_on_stale_ms=2000
levels=1
level_step_ticks=1

published counts the updates the control socket sent, applied is the sequence number of the last one the engine has applied. While they differ, the values are the ones the engine is about to apply (values=published). The status file carries the same numbers and the source of the last update (fastmm-top, --json under params; Status file), and the journal records each update with its origin and source (Journal format).

A strategy that brings its own publisher (LiveStrategy::set_params) numbers its updates itself: its replies carry no seq, --wait cannot confirm them and params shows no values.

Flatten¶

flatten is run by the engine, not the strategy, which may be what broke. It

stops quoting in its scope, pulls those quotes and cancels the working orders there (with --instrument only that instrument; without it the whole session);
every [engine] flatten_interval_ms (default 500) looks at the position of each instrument in scope and, when one is not flat and has no slice in flight, sends a reduce-only IOC limit order for the whole remaining position, priced --max-slippage-bps (default [engine] flatten_slippage_bps, 25) through the touch: selling at best_bid - bps, buying at best_ask + bps, rounded back to the tick;
ends when every instrument in scope is flat.

Only one slice per instrument is out at a time, so a flatten cannot sell the same position twice. A slice is capped at [risk] max_order_qty, so a large position leaves over several sweeps.

A flatten order is exempt from [risk] max_position (a reduce-only order exists to get under it) and from self-trade prevention (the flatten cancelled the orders of its scope first; a cancel the venue never acknowledges must not leave the position on). Everything else applies: the price collar, the fat-finger band, max_order_qty, max_order_notional, min_notional and the rate limit. A slice a limit refuses is retried at the next sweep. If the collar is narrower than the slippage, no slice can pass and the flatten never finishes; widen it with limits price_collar_bps=....

Progress is in the status file and in fastmm-ctl status:

flatten    state=Working instruments_left=1 orders=3

fastmm-top shows FLATTENING (1 left) next to the state, and Prometheus gets fastmm_flatten_state, fastmm_flatten_instruments_left and fastmm_flatten_orders_total.

When it cannot fill¶

Case	What happens
A slice is rejected (risk, rate limit, the venue) or does not fill	The next sweep reprices against the current touch and tries again.
No valid book for an instrument	Nothing is sent for it; the sweeps keep running and it resumes when the book is back.
What is left is below the lot or `min_qty`	No order can move it. The flatten logs `flatten leaves <qty> on <symbol>: below the tradable minimum` and treats that instrument as done.
`[engine] flatten_timeout_ms` (default 60000) elapses with a position left	The flatten gives up: `state=TimedOut`, an ERROR line naming how many instruments still hold a position, no further orders. Quoting stays off in its scope. Widen the slippage or the collar and run `flatten` again, or `resume` to give the session back to the strategy.
`flatten_timeout_ms = 0`	No deadline: it keeps trying until it is flat or an operator stops it.

resume (without a scope) stops a running flatten (state=Stopped) and hands the instruments back to the strategy. kill does not stop it: the kill switch refuses new orders, so the flatten's slices are rejected until you unkill. Flatten first, then kill.

A restart during a flatten¶

A flatten is not persisted and a restart does not resume it: the shutdown cancels the slice in flight, the new session starts quoting as [engine] quoting_enabled says, and the position is still there. Run flatten again if needed. The previous session's journal holds the whole story: tools/journal_dump.py prints the Flatten command, every sweep and every order it sent.

Behind a gateway¶

A strategy attached to fastmm-gateway has its own socket and takes every command above. The gateway has one too, fastmm-ctl --gateway <name>, whose pull, resume, kill and clear-kill act on every attached strategy or the account (Run behind a gateway).

Scope of the control socket¶

Change anything that is not a strategy parameter or a risk limit. Instruments, venues, threads, ring sizes and the strategy itself need a restart; ControlCommand::Reload is not implemented.
Cancel one order, or place one. The control plane works in scopes (session, venue, instrument), not in single orders.
Reach the engine from another host. The socket is local; tunnel over SSH.
Undo a fill. flatten trades the position away at the market; it does not restore the PnL.

Next: Kill switch and shutdown.