1 / 10
Explore
Engineering
Explore
Insights from a CppCon 2026 talk by Kris Jusiak, Lead Software Engineer at Citadel Securities
Insights from a CppCon 2026 talk by Kris Jusiak, Lead Software Engineer at Citadel Securities
A trading system can be fast on average and still be slow on the event that matters. In low-latency trading, performance doesn’t just depend on how quickly a system runs. How quickly and predictably that system reacts when a market update demands action is also important.
When that update arrives, the system must process new information, update its order book, evaluate a trading decision and potentially submit an order or cancellation, all within an extremely tight time window.
Raw speed tells only part of the story. A system that performs exceptionally well on average can still suffer costly delays when an actionable event encounters an unfavorable hardware state.
That’s why the latency of the critical order-generation path matters more than a headline throughput number or average execution time. Being fast most of the time is not enough. The challenge is delivering consistently low latency, even when execution conditions aren’t favorable.
So how can you improve the response that matters if you can’t accurately measure what slows it down?
Market making combines small edges, frequent decisions and quotes on both sides of the market. A bid is a price at which to buy; an ask is a price at which to sell. The difference between them is the spread. Capturing that spread repeatedly is the simplified economic picture behind the opening illustration.
A trading system can be fast on average and still be slow on the event that matters. In low-latency trading, performance doesn’t just depend on how quickly a system runs. How quickly and predictably that system reacts when a market update demands action is also important.
When that update arrives, the system must process new information, update its order book, evaluate a trading decision and potentially submit an order or cancellation, all within an extremely tight time window.
Raw speed tells only part of the story. A system that performs exceptionally well on average can still suffer costly delays when an actionable event encounters an unfavorable hardware state.
That’s why the latency of the critical order-generation path matters more than a headline throughput number or average execution time. Being fast most of the time is not enough. The challenge is delivering consistently low latency, even when execution conditions aren’t favorable.
So how can you improve the response that matters if you can’t accurately measure what slows it down?
Market making combines small edges, frequent decisions and quotes on both sides of the market. A bid is a price at which to buy; an ask is a price at which to sell. The difference between them is the spread. Capturing that spread repeatedly is the simplified economic picture behind the opening illustration.
Figure 1. Market making—this isn’t a record of trading performance.
Figure 1. Market making—this isn’t a record of trading performance.
Individual outcomes vary, while the blue cumulative line rises. The lesson isn’t that every trade earns the spread but that many small decisions contribute to an aggregate result, and the quality and timing of those decisions have an impact. Inventory, changing prices and execution costs remain part of the trading problem; the illustration is deliberately simplified.
Consider an order book with a best bid of 10.00 and a best ask of 10.01. A new market event can change whether those prices are still appropriate. To update a quote or withdraw an outdated order, the software must reach a decision and place the resulting message on the order-entry path. That’s the response timing we need to understand.
The objective is to measure the path that acts on market information under the conditions in which it must act. Follow that path from application code to processor behavior, then test whether an optimization improves it.
Latency measures how long one response takes. Throughput measures how many operations finish per unit of time. A backtest may be judged by how many historical ticks it processes per second. A live trading path must also respond quickly to a particular tick. Both measures are important, but they’re not interchangeable.
Trading: Latency (ns/op)
Individual outcomes vary, while the blue cumulative line rises. The lesson isn’t that every trade earns the spread but that many small decisions contribute to an aggregate result, and the quality and timing of those decisions have an impact. Inventory, changing prices and execution costs remain part of the trading problem; the illustration is deliberately simplified.
Consider an order book with a best bid of 10.00 and a best ask of 10.01. A new market event can change whether those prices are still appropriate. To update a quote or withdraw an outdated order, the software must reach a decision and place the resulting message on the order-entry path. That’s the response timing we need to understand.
The objective is to measure the path that acts on market information under the conditions in which it must act. Follow that path from application code to processor behavior, then test whether an optimization improves it.
Latency measures how long one response takes. Throughput measures how many operations finish per unit of time. A backtest may be judged by how many historical ticks it processes per second. A live trading path must also respond quickly to a particular tick. Both measures are important, but they’re not interchangeable.
Trading: Latency (ns/op)
The sequence matters: receive market data, update the book, evaluate the strategy and send an order when the strategy calls for one. So does the comment “Signal is rare.” The common no-action path can run continuously while the order-producing path receives little exercise.
Backtesting: Throughput (ticks/sec)
for day in date_range(date(2000, 1, 1), date(2026, 10, 20)):
for md in market_data(["SPY"], day):
strategy.on_data(md)
The replay loop is designed to work through a dataset. It can benefit from regular access patterns, warm caches and overlapping work across operations. Dividing elapsed time by the number of ticks gives an average processing cost for that experiment, not necessarily the response latency of a single live event.
The sequence matters: receive market data, update the book, evaluate the strategy and send an order when the strategy calls for one. So does the comment “Signal is rare.” The common no-action path can run continuously while the order-producing path receives little exercise.
Backtesting: Throughput (ticks/sec)
for day in date_range(date(2000, 1, 1), date(2026, 10, 20)):
for md in market_data(["SPY"], day):
strategy.on_data(md)
The replay loop is designed to work through a dataset. It can benefit from regular access patterns, warm caches and overlapping work across operations. Dividing elapsed time by the number of ticks gives an average processing cost for that experiment, not necessarily the response latency of a single live event.
Figure 2. Latency and inverse throughput for the same function—these are illustrative microbenchmark results, not trading-system timings.
Figure 2. Latency and inverse throughput for the same function—these are illustrative microbenchmark results, not trading-system timings.
Once the timing requirements for a given strategy are clear, hardware selection depends upon the workload. The illustration below places ASICs, FPGAs, CPUs, GPUs, TPUs and NPUs on a conceptual latency-to-throughput spectrum. Treat it as a design prompt, not a universal ranking: execution time still depends on the task, data movement and surrounding system.
Once the timing requirements for a given strategy are clear, hardware selection depends upon the workload. The illustration below places ASICs, FPGAs, CPUs, GPUs, TPUs and NPUs on a conceptual latency-to-throughput spectrum. Treat it as a design prompt, not a universal ranking: execution time still depends on the task, data movement and surrounding system.
Figure 3. Hardware alternatives: ASIC, FPGA, CPU, GPU, TPU and NPU.
Figure 3. Hardware alternatives: ASIC, FPGA, CPU, GPU, TPU and NPU.
An ASIC implements specialized logic; an FPGA provides reconfigurable hardware. A CPU offers general-purpose control flow and a flexible software development model. GPUs and tensor or neural accelerators emphasize parallel computation for suitable workloads. Efficiently completing a large batch doesn’t automatically make a device the best choice for the dependent steps of one order-generation event.
Three practical considerations accompany that choice. Time to market includes implementation effort and the ability to change a strategy. Knowing the deployment hardware, including a collocated environment, narrows the machine and topology that must be understood. Ease of optimization depends on how directly the workload maps to the platform and how measurable its behavior is. Evaluate hardware capability, engineering effort and response time together.
Data access, branch prediction and instruction supply are central performance constraints on modern CPUs. Even a short function can wait on a dependent load, recover from an incorrect prediction or stall while the frontend obtains the next instructions.
An ASIC implements specialized logic; an FPGA provides reconfigurable hardware. A CPU offers general-purpose control flow and a flexible software development model. GPUs and tensor or neural accelerators emphasize parallel computation for suitable workloads. Efficiently completing a large batch doesn’t automatically make a device the best choice for the dependent steps of one order-generation event.
Three practical considerations accompany that choice. Time to market includes implementation effort and the ability to change a strategy. Knowing the deployment hardware, including a collocated environment, narrows the machine and topology that must be understood. Ease of optimization depends on how directly the workload maps to the platform and how measurable its behavior is. Evaluate hardware capability, engineering effort and response time together.
Data access, branch prediction and instruction supply are central performance constraints on modern CPUs. Even a short function can wait on a dependent load, recover from an incorrect prediction or stall while the frontend obtains the next instructions.
Figure 4. AMD Zen5. Source: Dade Murphy, High-Res Die Shots Emerge of AMD Granite Ridge “Zen 5” CCD, ThinkComputers.org, October 7, 2024, https://thinkcomputers.org/high-res-die-shots-emerge-of-amd-granite-ridge-zen-5-ccd/
Figure 4. AMD Zen5. Source: Dade Murphy, High-Res Die Shots Emerge of AMD Granite Ridge “Zen 5” CCD, ThinkComputers.org, October 7, 2024, https://thinkcomputers.org/high-res-die-shots-emerge-of-amd-granite-ridge-zen-5-ccd/
These structures interact. If a price comparison depends on a load that misses in cache, the processor can’t resolve the comparison until the value arrives. If it then predicts the branch incorrectly, recovery adds another delay. Enlarging or rearranging the code can also change instruction-cache behavior without changing the algorithm.
These structures interact. If a price comparison depends on a load that misses in cache, the processor can’t resolve the comparison until the value arrives. If it then predicts the branch incorrectly, recovery adds another delay. Enlarging or rearranging the code can also change instruction-cache behavior without changing the algorithm.
Figure 5. Cache-access cost at the stated 4 GHz; one cycle is 0.25 ns.
Figure 5. Cache-access cost at the stated 4 GHz; one cycle is 0.25 ns.
Control the noise; don’t let the noise control your decisions. The untuned and tuned results below illustrate the difference between a variable run and a much tighter one. The goal is to separate the code’s behavior from avoidable experimental disturbance, not to discard inconvenient observations.
Control the noise; don’t let the noise control your decisions. The untuned and tuned results below illustrate the difference between a variable run and a much tighter one. The goal is to separate the code’s behavior from avoidable experimental disturbance, not to discard inconvenient observations.
Figure 6. Untuned and tuned measurements based on RDTSC.
Figure 6. Untuned and tuned measurements based on RDTSC.
Low-latency trading systems require careful system-level tuning to minimize execution variability and achieve consistent performance, including CPU isolation and thread pinning, NUMA-aware memory placement, cache optimization and appropriate kernel and network bypass configurations. Hardware and operating system settings, e.g., SMT, frequency scaling and memory management, should be evaluated and optimized for latency-sensitive workloads. These optimizations must be implemented in a controlled environment and validated against the production configuration to ensure reliable, deterministic performance.
Hardware counters help turn an explanation into a testable hypothesis. Top-down microarchitecture analysis classifies pipeline slots as retiring, bad speculation, frontend-bound or backend-bound.
Low-latency trading systems require careful system-level tuning to minimize execution variability and achieve consistent performance, including CPU isolation and thread pinning, NUMA-aware memory placement, cache optimization and appropriate kernel and network bypass configurations. Hardware and operating system settings, e.g., SMT, frequency scaling and memory management, should be evaluated and optimized for latency-sensitive workloads. These optimizations must be implemented in a controlled environment and validated against the production configuration to ensure reliable, deterministic performance.
Hardware counters help turn an explanation into a testable hypothesis. Top-down microarchitecture analysis classifies pipeline slots as retiring, bad speculation, frontend-bound or backend-bound.
Figure 7. Top-down microarchitecture analysis. Percentages describe pipeline slots, not fractions of the event’s elapsed latency.
Figure 7. Top-down microarchitecture analysis. Percentages describe pipeline slots, not fractions of the event’s elapsed latency.
In this example, 59.2% of backend-bound slots call for a closer distinction between memory pressure and execution-resource limits. That doesn’t mean that removing memory accesses removes 59.2% of the response time. Test the suspected cause with a controlled change, then measure the event again.
Linux provides software events, tracepoints, dynamic probes and hardware performance counters. The diagrams below show how observability extends from applications and system calls to the processor and memory system.
In this example, 59.2% of backend-bound slots call for a closer distinction between memory pressure and execution-resource limits. That doesn’t mean that removing memory accesses removes 59.2% of the response time. Test the suspected cause with a controlled change, then measure the event again.
Linux provides software events, tracepoints, dynamic probes and hardware performance counters. The diagrams below show how observability extends from applications and system calls to the processor and memory system.
Figure 8. Linux perf_events event sources. Source: Brendan Gregg, Linux Extended BPF (eBPF) Tracing Tools, BrendanGregg.com, n.d., https://www.brendangregg.com/ebpf.html
Figure 8. Linux perf_events event sources. Source: Brendan Gregg, Linux Extended BPF (eBPF) Tracing Tools, BrendanGregg.com, n.d., https://www.brendangregg.com/ebpf.html
Figure 9. Linux perf/bcc/BPF events. Source: Brendan Gregg, Linux Extended BPF (eBPF) Tracing Tools, BrendanGregg.com, n.d., https://www.brendangregg.com/ebpf.htm
Figure 9. Linux perf/bcc/BPF events. Source: Brendan Gregg, Linux Extended BPF (eBPF) Tracing Tools, BrendanGregg.com, n.d., https://www.brendangregg.com/ebpf.htm
A sampled profile shows where execution was observed. It doesn’t record the start and finish of every order-producing event. A common path can dominate the profile while a short, commercially relevant branch doesn’t receive enough observations to characterize its latency. Sampling is useful for aggregate hotspots, but it’s not a substitute for event-level latency measurement.
A sampled profile shows where execution was observed. It doesn’t record the start and finish of every order-producing event. A common path can dominate the profile while a short, commercially relevant branch doesn’t receive enough observations to characterize its latency. Sampling is useful for aggregate hotspots, but it’s not a substitute for event-level latency measurement.
Suppose only 0.1% of updates produce an order. A percentile calculated across all updates can conceal that entire event class: even the overall p99 need not describe order generation. Separate order-producing updates, cancellations and no-action updates, then measure the distribution for each. The 0.1% figure is illustrative, not production frequency.
The source comparison below evaluates CPU mechanisms for this rare hot-path measurement problem. Its “HFT-friendly” assessment is not a blanket rejection of sampling, branch records or profile-guided optimization.
Availability, privileges and semantics depend on the processor and operating system. The dash is the source’s entry, not a claim about all present or future AMD tracing capabilities. PGO means “profile-guided optimization.”
Read Performance-Monitoring Counters (RDPMC) can be used for measuring latency and performance monitoring unit (PMU) counters to explain the work inside a defined region. Order the reads correctly and quantify their overhead. A timestamp-counter tick is not necessarily a core clock cycle; conversion to nanoseconds requires the appropriate counter frequency. Instrumentation can materially perturb a short, measured region.
Intel Processor Trace (PT) provides a different kind of evidence: compressed control-flow information that can be decoded against the matching binary. It can reveal the path taken by an unusual event without requiring a sampling interrupt to land inside that path. Correlating the trace with event timings helps distinguish a different execution path from a slower execution of the same path.
Suppose only 0.1% of updates produce an order. A percentile calculated across all updates can conceal that entire event class: even the overall p99 need not describe order generation. Separate order-producing updates, cancellations and no-action updates, then measure the distribution for each. The 0.1% figure is illustrative, not production frequency.
The source comparison below evaluates CPU mechanisms for this rare hot-path measurement problem. Its “HFT-friendly” assessment is not a blanket rejection of sampling, branch records or profile-guided optimization.
Availability, privileges and semantics depend on the processor and operating system. The dash is the source’s entry, not a claim about all present or future AMD tracing capabilities. PGO means “profile-guided optimization.”
Read Performance-Monitoring Counters (RDPMC) can be used for measuring latency and performance monitoring unit (PMU) counters to explain the work inside a defined region. Order the reads correctly and quantify their overhead. A timestamp-counter tick is not necessarily a core clock cycle; conversion to nanoseconds requires the appropriate counter frequency. Instrumentation can materially perturb a short, measured region.
Intel Processor Trace (PT) provides a different kind of evidence: compressed control-flow information that can be decoded against the matching binary. It can reveal the path taken by an unusual event without requiring a sampling interrupt to land inside that path. Correlating the trace with event timings helps distinguish a different execution path from a slower execution of the same path.
PT doesn’t provide an exact timestamp for every instruction, and ordinary control-flow tracing doesn’t capture every data value. Check processor support, decoder errors, lost trace data and measurement overhead. Preserve the exact executable and mapping information needed for decoding. Trace explains execution; a separately defined timing boundary establishes response latency.
To avoid measurement biases, consider hardware effects. Looking at source code, assembly, address, encoding and CPU helps explain why a source-level change can behave differently after compilation and why a binary-layout change can affect timing without changing the algorithm.
PT doesn’t provide an exact timestamp for every instruction, and ordinary control-flow tracing doesn’t capture every data value. Check processor support, decoder errors, lost trace data and measurement overhead. Preserve the exact executable and mapping information needed for decoding. Trace explains execution; a separately defined timing boundary establishes response latency.
To avoid measurement biases, consider hardware effects. Looking at source code, assembly, address, encoding and CPU helps explain why a source-level change can behave differently after compilation and why a binary-layout change can affect timing without changing the algorithm.
Figure 10. Example source/assembly/address/encoding.
Figure 10. Example source/assembly/address/encoding.
For example, a load from book[lvl] introduces a data-cache dependency. The comparison feeds a conditional jump, creating a potential prediction dependency. Function alignment, padding and code placement affect instruction delivery; the stack frame and return sequence add their own accesses and control transfers. Static data layout determines where the referenced values reside.
A performance result must therefore identify the compiler, options and binary being measured. Inlining, instruction selection, alignment and function ordering can change the limiting resource. A source change is only an optimization when the resulting program remains correct, and the relevant timing improves.
A trace shows what happened in a recorded run. A benchmark must ask what else could happen. Repeatedly testing one convenient input can miss a path that generates an order, enters a risk check or performs an instrument lookup. Symbolic benchmarking connects input selection to the code those inputs cause to execute.
For example, a load from book[lvl] introduces a data-cache dependency. The comparison feeds a conditional jump, creating a potential prediction dependency. Function alignment, padding and code placement affect instruction delivery; the stack frame and return sequence add their own accesses and control transfers. Static data layout determines where the referenced values reside.
A performance result must therefore identify the compiler, options and binary being measured. Inlining, instruction selection, alignment and function ordering can change the limiting resource. A source change is only an optimization when the resulting program remains correct, and the relevant timing improves.
A trace shows what happened in a recorded run. A benchmark must ask what else could happen. Repeatedly testing one convenient input can miss a path that generates an order, enters a risk check or performs an instrument lookup. Symbolic benchmarking connects input selection to the code those inputs cause to execute.
Symbolic execution treats unknown values as variables and records constraints along feasible paths. Solving those constraints yields concrete register and memory values for the measurement harness. In this example, a useful test set must exercise rejection, buying, selling and no action, rather than repeatedly measuring only the no-trade case. The solver discovers cases; native execution on the intended CPU determines their cost.
The optimized binary remains the target. A compiler can replace branches with conditional moves, combine tests or change layout. Valid trading-function tests also include object invariants, pointer ownership and permitted ranges, not just integers that satisfy a condition. Explicitly bound loops, external calls and path counts. Discovering several paths doesn’t prove exhaustive coverage. Setting the hardware state must be part of the setup process for bias-free benchmarking.
The recurring “Signal is rare” comment has a microarchitectural consequence. A processor predicts control flow before every condition is fully resolved. An incorrect prediction requires it to discard speculative work and recover. The frequently repeated no-order path and the transition to an order-producing path deserve separate experiments.
Symbolic execution treats unknown values as variables and records constraints along feasible paths. Solving those constraints yields concrete register and memory values for the measurement harness. In this example, a useful test set must exercise rejection, buying, selling and no action, rather than repeatedly measuring only the no-trade case. The solver discovers cases; native execution on the intended CPU determines their cost.
The optimized binary remains the target. A compiler can replace branches with conditional moves, combine tests or change layout. Valid trading-function tests also include object invariants, pointer ownership and permitted ranges, not just integers that satisfy a condition. Explicitly bound loops, external calls and path counts. Discovering several paths doesn’t prove exhaustive coverage. Setting the hardware state must be part of the setup process for bias-free benchmarking.
The recurring “Signal is rare” comment has a microarchitectural consequence. A processor predicts control flow before every condition is fully resolved. An incorrect prediction requires it to discard speculative work and recover. The frequently repeated no-order path and the transition to an order-producing path deserve separate experiments.
The optional warm-up path illustrates the presentation’s “Warm cache/branch” idea; it doesn’t describe a production implementation. It shouldn’t submit orders or alter trading state. It also consumes execution resources and can displace useful instructions or data, so measure any benefit.
N denotes no order; S denotes a signal. These are experimental sequences, not guaranteed prediction outcomes or production frequencies. To test the rare transition, use the preceding no-order events as preparation and time the final order-producing event separately. Hold cache conditions and the final input constant when isolating the effect of history.
Changing values doesn’t necessarily make a branch unpredictable. A periodic pattern may be learned, and random inputs don’t guarantee a misprediction. A source-level if may not even become a machine-code branch. Inspect the binary and collect branch-miss counters alongside cycles and retired instructions.
Branchless code isn’t automatically faster, either. It may add loads, computation or dependencies the branch would have avoided. Compiler likelihood annotations influence compilation; they don’t directly program the hardware predictor. The test is event-level execution time under the histories the system must handle.
A tight benchmark loop does more than train control flow: it keeps its working set close to the core. A model parameter or order-entry object that’s hot in a benchmark may not remain hot after a long interval of unrelated market-data work. If the next instruction depends on that value, the load delay sits on the critical path.
The optional warm-up path illustrates the presentation’s “Warm cache/branch” idea; it doesn’t describe a production implementation. It shouldn’t submit orders or alter trading state. It also consumes execution resources and can displace useful instructions or data, so measure any benefit.
N denotes no order; S denotes a signal. These are experimental sequences, not guaranteed prediction outcomes or production frequencies. To test the rare transition, use the preceding no-order events as preparation and time the final order-producing event separately. Hold cache conditions and the final input constant when isolating the effect of history.
Changing values doesn’t necessarily make a branch unpredictable. A periodic pattern may be learned, and random inputs don’t guarantee a misprediction. A source-level if may not even become a machine-code branch. Inspect the binary and collect branch-miss counters alongside cycles and retired instructions.
Branchless code isn’t automatically faster, either. It may add loads, computation or dependencies the branch would have avoided. Compiler likelihood annotations influence compilation; they don’t directly program the hardware predictor. The test is event-level execution time under the histories the system must handle.
A tight benchmark loop does more than train control flow: it keeps its working set close to the core. A model parameter or order-entry object that’s hot in a benchmark may not remain hot after a long interval of unrelated market-data work. If the next instruction depends on that value, the load delay sits on the critical path.
Figure 11. Cache access reference cycles on Intel Sapphire Rapids.
Figure 11. Cache access reference cycles on Intel Sapphire Rapids.
The refresh pseudocode below makes the intended experiment explicit: touch a line for an L1-oriented condition, use flush and prefetch for an intermediate-cache condition, request demotion for a more distant cache or flush for a cold condition. Preparation belongs outside the measured region.
The refresh pseudocode below makes the intended experiment explicit: touch a line for an L1-oriented condition, use flush and prefetch for an intermediate-cache condition, request demotion for a more distant cache or flush for a cold condition. Preparation belongs outside the measured region.
Figure 12. Cache-line eviction.
Figure 12. Cache-line eviction.
Prefetch and demotion are hints. Hardware may choose a different residency outcome or ignore a hint; a fence doesn’t turn a hint into a placement guarantee. Detect support through the processor’s feature enumeration rather than the slide’s generation shorthand. In particular, CLDEMOTE support is not exclusive to the server generation named in the slide.
Once you’ve measured the same path under the same conditions, determine whether the candidate improved performance, or if the difference is ordinary run-to-run variation. The central limit theorem and a null-hypothesis test can help assess a mean comparison, but they don’t replace testing the adverse path.
The central limit theorem concerns averages, not the shape of individual latency observations. For sufficiently many independent, identically distributed observations with finite variance, the appropriately standardized sample mean approaches a normal distribution. When those assumptions are justified, this supports approximate uncertainty estimates for a mean comparison. It doesn’t make the underlying latency tail normal.
Define the mean difference as candidate minus baseline, so a negative value means faster. For a one-sided improvement test, the null hypothesis is that the candidate hasn’t reduced the mean: the difference is zero or positive. Compare matching paths and machine states, randomize the order of baseline and candidate runs and report the estimated difference with a confidence interval.
Prefetch and demotion are hints. Hardware may choose a different residency outcome or ignore a hint; a fence doesn’t turn a hint into a placement guarantee. Detect support through the processor’s feature enumeration rather than the slide’s generation shorthand. In particular, CLDEMOTE support is not exclusive to the server generation named in the slide.
Once you’ve measured the same path under the same conditions, determine whether the candidate improved performance, or if the difference is ordinary run-to-run variation. The central limit theorem and a null-hypothesis test can help assess a mean comparison, but they don’t replace testing the adverse path.
The central limit theorem concerns averages, not the shape of individual latency observations. For sufficiently many independent, identically distributed observations with finite variance, the appropriately standardized sample mean approaches a normal distribution. When those assumptions are justified, this supports approximate uncertainty estimates for a mean comparison. It doesn’t make the underlying latency tail normal.
Define the mean difference as candidate minus baseline, so a negative value means faster. For a one-sided improvement test, the null hypothesis is that the candidate hasn’t reduced the mean: the difference is zero or positive. Compare matching paths and machine states, randomize the order of baseline and candidate runs and report the estimated difference with a confidence interval.
Figure 13. Illustrative 95% confidence intervals for the difference in mean latency. Both estimates are -3 ns. The intervals are hypothetical, not measured performance results.
Figure 13. Illustrative 95% confidence intervals for the difference in mean latency. Both estimates are -3 ns. The intervals are hypothetical, not measured performance results.
Most importantly, a mean-latency comparison doesn’t establish a tail improvement or a worst-case bound. Retain individual action timings, examine the relevant tail percentiles and slowest observations, and repeat adverse-state tests. A faster average doesn’t justify a regression in the order-generation path you need to protect.
After isolating individual effects, combine them deliberately. What happens when an order-producing update needs cold data, encounters an unfamiliar branch pattern and reaches code in an unfavorable layout? Move beyond a comfortable steady state and test the same path when several conditions work against it.
Most importantly, a mean-latency comparison doesn’t establish a tail improvement or a worst-case bound. Retain individual action timings, examine the relevant tail percentiles and slowest observations, and repeat adverse-state tests. A faster average doesn’t justify a regression in the order-generation path you need to protect.
After isolating individual effects, combine them deliberately. What happens when an order-producing update needs cold data, encounters an unfamiliar branch pattern and reaches code in an unfavorable layout? Move beyond a comfortable steady state and test the same path when several conditions work against it.
Each group asks a different question. Code and function alignment change instruction placement; function order changes layout. Stack size and placement change the addresses used. Branch conditioning changes the preceding history or input sequence. The data- and instruction-cache fields request cold states, while the data and instruction TLB fields target address-translation effects.
Each group asks a different question. Code and function alignment change instruction placement; function order changes layout. Stack size and placement change the addresses used. Branch conditioning changes the preceding history or input sequence. The data- and instruction-cache fields request cold states, while the data and instruction TLB fields target address-translation effects.
Figure 14. Best, real and worst executions.
Figure 14. Best, real and worst executions.
The backend-bound share changes from 20% in the best scenario to 55% in the worst, indicating a change in the limiting resource, not a 2.75-times increase in latency. Read counters alongside the timing distribution. A different bottleneck can explain a result, but only the timing measurement establishes how much the response changed.
Finally, validate the result with realistic event ordering and arrival timing. A replay that injects the next event only after the previous one finishes can hide queueing when the system falls behind. A local benchmark explains a component; by itself, it doesn’t establish end-to-end behavior during a burst.
Following a trading response down to the machine shows why low-latency engineering spans several disciplines. Market behavior defines the event that matters. C++ and compiler decisions define the executable. The processor determines how that executable behaves in a particular state. Measurement determines whether a change helped.
At Citadel Securities, engineers work directly with traders and quantitative researchers. Public engineering roles span software, research infrastructure, reliability and FPGA development. The connection is practical because understanding the trading requirement and understanding the machine are parts of the same engineering problem.
The standard for an optimization is specific: the program remains correct, the event and timing boundary are defined, the hardware conditions are understood and repeatable evidence supports the improvement. Nanoseconds matter when they belong to the response the system needs to deliver.
Don’t optimize the number that’s easiest to report. Optimize the response the system needs to deliver.
The backend-bound share changes from 20% in the best scenario to 55% in the worst, indicating a change in the limiting resource, not a 2.75-times increase in latency. Read counters alongside the timing distribution. A different bottleneck can explain a result, but only the timing measurement establishes how much the response changed.
Finally, validate the result with realistic event ordering and arrival timing. A replay that injects the next event only after the previous one finishes can hide queueing when the system falls behind. A local benchmark explains a component; by itself, it doesn’t establish end-to-end behavior during a burst.
Following a trading response down to the machine shows why low-latency engineering spans several disciplines. Market behavior defines the event that matters. C++ and compiler decisions define the executable. The processor determines how that executable behaves in a particular state. Measurement determines whether a change helped.
At Citadel Securities, engineers work directly with traders and quantitative researchers. Public engineering roles span software, research infrastructure, reliability and FPGA development. The connection is practical because understanding the trading requirement and understanding the machine are parts of the same engineering problem.
The standard for an optimization is specific: the program remains correct, the event and timing boundary are defined, the hardware conditions are understood and repeatable evidence supports the improvement. Nanoseconds matter when they belong to the response the system needs to deliver.
Don’t optimize the number that’s easiest to report. Optimize the response the system needs to deliver.
