Design High-Throughput Stock Exchange

An exchange is a meeting point. Brokers and trading firms send it orders to buy or sell a listed instrument at some price, and the exchange decides, deterministically and in public, which orders trade with which. Everything else, from the order entry links to the market data feeds, exists to make that one decision fast, fair and provable.

Speed matters more here than in almost any other system you will design in an interview. Firms co-locate their servers in the exchange's data centre and measure the round trip from sending an order to seeing its acknowledgement in tens of microseconds. A garbage collection pause of a few milliseconds would be an incident. So would two replicas disagreeing on whether a trade happened.

The design that comes out of these constraints looks unusual: a single thread per group of instruments doing all the matching, an input log that defines the truth, standby machines that replay the same log in lockstep, and networking that bypasses the operating system kernel. This page walks through why each of those choices is the sensible one.

Assume a mid-sized equities venue: about 8,000 listed instruments, 400 member firms, and 2 billion order messages a day across a 6.5-hour session. Most messages are not trades. Market makers constantly cancel and replace quotes, so the ratio of messages to trades is about 50 to 1.

Low-latency firms use a fixed-layout binary protocol over TCP, in the style of NASDAQ OUCH. Every field sits at a known offset, so the gateway parses without branching or allocating.

Prices travel as integer ticks, never as floating point. A stock with a one-cent tick trading at 182.45 is sent as 18,245. That keeps comparisons exact and lets the engine use the price as an array index.

Slower clients and most brokers use FIX 4.4 or 5.0, a tag and value text protocol. A dedicated FIX gateway translates each message into the same internal binary form before it reaches the sequencer, so the engine never parses text.

The public feed is sent as UDP multicast, one message per book change, wrapped in packets with a session ID and a sequence number. Subscribers that miss a packet ask a retransmission server for the range, or rebuild from a periodic snapshot channel.

Every order follows the same path. A gateway terminates the firm's session, validates the message and converts it to the internal format. Pre-trade risk checks the firm's credit and the order's size and price against limits held in memory. The sequencer stamps it with the next number in the input log for its partition, writes it to a journal replicated on two machines, and only then passes it to the matching engine.

The matching engine is a single thread that reads sequenced events one at a time and updates the order books it owns. It is the only component that changes book state. Its outputs, acknowledgements, fills and book changes, go to an output publisher that sends private reports back through the gateways, public updates to the market data feeds, and a complete copy to clearing and to firms' drop-copy sessions.

A standby engine reads exactly the same sequenced input and produces exactly the same output, because the engine is deterministic. Nobody listens to the standby while the primary is healthy, but its output is compared with the primary's, which catches bugs and hardware faults. If the primary dies, the standby is already in the same state and takes over.

A book has two sides. Each side is a set of price levels, and each level is a first-in, first-out queue of resting orders. Three operations dominate: add an order at a level, cancel an order by its ID, and match against the best level on the other side. Cancels are by far the most common, often 80% or more of all messages, so they must be cheap.

The classic layout uses three structures. Price levels live in an array indexed by tick offset from a reference price, so finding the level for a price is arithmetic, not a search. Within each level, orders form an intrusive doubly linked list, where the next and previous pointers live inside the order record itself, so removing one order is constant time with no allocation. And a hash map from order ID to order record lets a cancel jump straight to its node.

Order records come from a pre-allocated pool and are sized to fit in one 64-byte cache line. Nothing is allocated during trading. When a level empties, the engine moves its best-price pointer to the next populated level, and a small bitmap over the price array makes that scan a few instructions.

Why not a balanced tree of price levels? A tree works and is a fine first answer, but every lookup chases pointers across memory, and each cache miss costs around 100 nanoseconds. For instruments whose price stays within a few thousand ticks of the reference during a day, the array wins comfortably. Instruments with very wide ranges can fall back to a tree or a sparse two-level array.

When an aggressive buy order arrives, the engine looks at the best ask. If the order's limit is at or above it, the two trade at the resting order's price, the quantities reduce, and fully filled orders leave the book. The engine repeats against the next resting order, then the next level, until the incoming order is filled or its limit no longer crosses. Whatever remains rests in the book, unless the order is immediate-or-cancel, in which case it is cancelled.

Other order types are rules wrapped around this loop. Fill-or-kill checks available quantity up to the limit price before touching anything and rejects if there is not enough. Post-only rejects or reprices an order that would cross. Stop orders sit outside the book in a trigger table keyed by price, and the engine releases them as normal orders when a trade prints through their trigger.

The engine is a pure function of its input log. Given the same sequence of events, it must produce the same trades, the same IDs and the same feed messages, byte for byte. That rule bans several things you would use without thinking in normal services: reading the wall clock inside the engine, iterating a hash map in an order that depends on memory addresses, using threads that might interleave differently, or making any network call. Time comes from the sequencer, stamped onto each event before it enters the log.

What you get in return is enormous. Replication becomes "run the same log on another machine". Recovery becomes "load a snapshot and replay from there". Debugging a production incident becomes "replay yesterday's log on a laptop and step through the exact trade". Regulators can ask how a trade happened and receive a precise answer.

A multi-threaded engine needs locks or atomic operations on the books, and both introduce contention and timing variation. A single thread per partition does each operation in a few hundred nanoseconds with no coordination at all. Partitioning by instrument gives parallelism across the venue, because orders for one instrument never need to see another instrument's book.

Between components, events move through ring buffers in the style of the LMAX Disruptor: a pre-allocated array with one writer and sequence counters for each reader. Writers and readers never take a lock, they only publish and read sequence numbers. Each counter sits on its own cache line, so two cores updating neighbouring counters do not keep invalidating each other's caches. Threads busy-spin on dedicated cores rather than sleeping, because waking a sleeping thread costs microseconds.

A normal network receive involves an interrupt, a copy into kernel memory, the TCP stack, a system call and another copy. That is several microseconds before your code sees a byte. Gateways and engine hosts use kernel-bypass network stacks, such as DPDK or vendor libraries like Solarflare's, which map the network card's queues straight into user space. The application polls the card directly and handles packets without any system call.

Hardware timestamps from the network card record when each packet hit the wire. They are used for latency monitoring and for regulatory clock-sync requirements, never for ordering decisions inside the engine, which uses the sequencer's order only.

An order is not acknowledged until its sequenced event is persisted on two machines. Writes go to NVMe through a memory-mapped or direct I/O journal, and the replica receives the same bytes over a dedicated link. This adds a few microseconds, and it is the price of never losing an acknowledged order.

The standby engine consumes the same released events. When monitoring declares the primary dead, a single arbiter promotes the standby and fences the old primary by incrementing an epoch that the publisher checks on every output. Any output stamped with an older epoch is dropped, so a primary that was only slow, not dead, cannot publish conflicting results. The standby resumes from the next sequence number, and because it already processed every released event, no order is reordered or replayed to clients.

Recovery from cold uses snapshots. Every few minutes the engine writes its full state, marked with the last sequence it applied, without pausing: the standby takes the snapshot so the primary never stops. Restoring loads the latest snapshot and replays the journal from that sequence onwards.

Sending each subscriber its own TCP stream would mean thousands of copies of every update, all slightly out of step. Instead the exchange sends one stream as UDP multicast, and switches copy packets to every subscribed port at the same moment. That makes the public view simultaneous for everyone in the hall.

UDP can drop packets, so every packet carries a sequence number. The feed is sent twice, on independent A and B lines through different switches, and subscribers take whichever copy arrives first. If both are lost, the subscriber requests the missing range from a retransmission server, or rejoins from a snapshot channel that cycles through every book. The publisher also groups several messages into one packet when they are produced within a microsecond or two, which cuts packet rate during bursts without adding meaningful delay.

Fairness at an exchange means equal access, not equal outcomes. Equal cable lengths in the co-location hall, the same gateway hardware and software for every firm, and ordering by sequencer arrival are the basics. Some venues go further: IEX adds a fixed delay of about 350 microseconds to incoming orders using a coil of fibre, so that firms cannot react to price changes elsewhere before the exchange updates its own pegged orders. Others run frequent batch auctions instead of continuous matching. Mention these as design options with clear trade-offs, not as obvious improvements.

During the pre-open, orders collect without matching. At the open, the engine computes one clearing price for the whole book. For every candidate price, it adds up how much buyers will buy at or above it and how much sellers will sell at or below it, and picks the price where the executable quantity is largest. Ties are broken by the smallest leftover imbalance, then by closeness to a reference price such as the previous close. Every crossing order trades at that single price. The engine publishes indicative prices and imbalances during the pre-open so participants can react.

In this example the auction prints at 182.50, the price where 6,400 shares can trade. The remaining 900 shares of sell interest rest in the book when continuous trading begins.