the last question

Physics Go Brrr

Feb 17, 2026

Training my steering controller required new simulation data before each policy update. I batched the simulator to reduce that cost.

An early serial run took about 50 minutes for 5,000 routes on a MacBook. The later GPU version took about eight seconds, plus three seconds to update the policy. These recorded timings span different hardware and implementations, not a controlled software speedup. The serial baseline was my training path. The official evaluator already used 16 workers.

Batching routes

The physics model predicts lateral acceleration from steering commands, road state and recent acceleration. It outputs scores for 1,024 acceleration tokens, each representing an acceleration value. The simulator converts scores to probabilities, samples a token and feeds its acceleration into the next call. A roughly 600-sample route needs about 580 calls after its initial 20-sample history.

Different routes are independent. The batched simulator advances them together, passing one array of routes to the model at each timestep. For 5,000 equal-length routes, roughly 2.9 million model invocations become 580 larger ones. Batching reduces call overhead and gives the hardware more parallel work. It does not remove the arithmetic.

The model’s state input has shape (N, 20, 4) for N routes, 20 recent steps and four values per step. Those values are steering, road-bank acceleration, speed and forward acceleration. Acceleration tokens form a separate (N, 20) array. I preallocated histories indexed by route and timestep so each call could read the recent window and write its result into the next position.

Distributing the work

Before moving to a GPU, I connected a MacBook Pro and Air over Thunderbolt.

The first version started a remote worker over SSH each epoch. A persistent connection kept it running between updates. Equal workloads took 42 seconds on the Pro and 58 on the Air, so I assigned more routes to the faster machine.

The setup grew to three machines. Each collected experience with the same policy and returned it for one update. I later moved to a rented RTX 5060 Ti.

Keeping data on the GPU

Moving model inference to CUDA still left input construction and history updates on the CPU. Every timestep copied data between them.

IOBinding let ONNX Runtime read and write GPU buffers directly. I moved sampling and histories there too, then rewrote the controller’s input construction in PyTorch. The recorded controller time fell from about 9.5 seconds to 1.3 seconds.

Outside that loop, CSV files were still being parsed every epoch. A cache loaded them once and selected rows for each batch. Collected observations and actions also stayed in GPU tensors for the policy update, avoiding another copy of the training data.

TensorRT with FP16 enabled reduced recorded simulator time from about ten seconds to five. The eight-second rollout excluded initial CSV loading and engine compilation.

Checking the result

The rewrite also had to preserve numerical behavior. The reference converts acceleration to discrete tokens using float64 bin values. Rounding those values through float32 can change a later token. I retained float64 bins and matched the tokenizer’s boundary convention.

Random draws also needed to match. The reference seeds from the full route path and consumes draws during warmup, even when it discards the predictions.

A saved CPU comparison using the exp052 controller on 100 routes reported 42.4748 for both simulators, with zero per-route differences at the printed precision.

The TensorRT version averaged 42.4088 against the reference’s 42.4748. Mean absolute per-route difference was 0.5115, with a maximum of 4.0361. A similar average did not mean matching trajectories. I kept final evaluation on the reference simulator.

The later exp055 training run used a different GPU, an RTX 5090. Its 5,000 epochs took about 10.41 hours at USD 0.301 an hour, or about USD 3.14 for that run.