Low Latency in Simulation Middleware: Practical Lessons

In flight simulation, latency shows up as a physical mismatch between the pilot’s input and the system’s response, usually visuals or motion. When those signals don’t align, the simulator feels out of sync even if the frame rate is high.

I’ve spent a lot of time building middleware that translates packets between different protocols. What I learned is that performance usually comes down to predictability, not raw speed. A stable processing time is more important than being “fast on average”.

Latency and jitter

People talk about latency a lot, but jitter is usually the bigger problem. A pilot can adapt to a steady 20 ms lag. They can’t adapt to a delay that jumps between 10 ms and 50 ms.

When I build middleware, I try to make the processing time as deterministic as possible. And in my field, middleware with latency larger than 1 ms is considered slow.

Where delay comes from

I split delay into two areas.

The first is network and OS delay. This is time lost before your application even sees the packet. Kernel buffering, driver overhead, scheduling, and that kind of stuff. You have limited control here, outside of OS tuning and how you capture traffic.

The second is application delay. This is time spent inside your code: parsing, coordinate conversion, and thread synchronization. This is where you can adjust your code to avoid lag.

Reducing memory movement

It is very easy to accidentally move data too much. The common pattern is: receive a UDP buffer, copy it into a struct, pass it to a function that copies it into a class, then copy it again into an output buffer.

Each memcpy costs time, but the bigger issue is timing fluctuation. Under load, those extra copies don’t just add delay. They add inconsistency.

In my middleware, I try to process in place, or use a single pre-allocated buffer for the whole pipeline. I want the packet to arrive, get touched once, and leave.

Selective parsing

You don’t always need to decode the entire packet. If you receive a big CIGI or DIS update but you only need the ownship position, decoding everything is wasted CPU.

I treat it like skimming a letter to confirm what it is, instead of reading every line.

I found that adding early exit rules, like checking the packet ID and dropping it before the parsing logic starts, stabilized CPU load a lot. If you don’t need a packet, touch it for the shortest time possible.

Avoiding allocations in the time-sensitive path

Calling new or malloc inside the main loop is a classic source of jitter. Most of the time allocations are fast. Sometimes they spike. This is because the allocator may do extra work, like searching, syncing, or reclaiming memory. You don’t control when that happens, and it shows up as a random timing hit.

Fixed-size, pre-allocated buffers are boring, but they remove this variable. For middleware, boring is good.

Backlog and outdated data

This is a real-time problem. If traffic spikes and you can’t keep up, a queue builds up.

If you then try to process every packet in that queue, you end up forwarding outdated data. Even when the spike ends, your middleware is still behind real time because it’s busy catching up.

In simulation, it is usually better to drop older state updates and jump to the newest one than to process everything in order. Sequence integrity matters for some message types, but state updates are not one of them.

Compromises in real-time design

There is no pure win. Every gain has a cost.

If you want lower receive delay, you can spin in a thread and catch the packet the moment it hits the NIC. This works, but it burns a CPU core at 100%, and it can create scheduling problems for the rest of the system.

If you want more Bandwidth, batching packets can help, but it ruins timing. This is because the first packet in the batch is forced to wait for the last one. In flight sims, consistent spacing is usually more important than total capacity.

Reliability is not one idea either. Some data is disposable. A position update can be dropped because a new one is coming soon. A control command like a gear toggle cannot be dropped because it might be sent once. In practice, this means different message types need different handling paths.

What worked for me

I don’t trust averages. I measure worst-case behavior, like p99, because averages hide the spikes that pilots actually feel.

I filter early. I check the port or message ID before doing any real work.

And I keep the hot path predictable. I want the code to do the same amount of work every time a packet arrives. If one path is heavier than another, that difference becomes jitter. In a simulator, jitter is what people notice first.