ML-Physics: Teaching a Neural Net to Fake Physics

devlog
ml
gamedev
Building a 2D physics engine that uses classical solvers to train a neural net to replace itself.
Published

June 25, 2026

So I don’t talk about this much on here, but I have been quietly building something I think is actually kind of cool — a 2D physics engine where the classical solver trains a neural net to eventually replace the expensive parts of itself. The long-term goal is to get something that can run KSP or Space Engine scale simulations on mid-range hardware. That’s the dream anyway.

Let me explain what I mean, because I feel like this gets misunderstood fast.

What Even Is a Constraint Solver?

A classical rigid body physics engine works by defining constraints — rules that the simulation has to satisfy every frame. Things like “these two boxes cannot overlap” or “this joint has to stay connected.” Every frame, the solver looks at all the violations and applies corrective impulses to the bodies to fix them. The specific method I am using is called a sequential impulse solver, which is what Box2D uses and what Erin Catto has been writing about for years in his GDC papers.1

The cool thing about this approach is it is deterministic. Same inputs, same outputs, every time. That matters a lot for what comes next.

Figure 1: Every frame, in order, no exceptions.

So Where Does the ML Come In?

Here is the thing — constraint solvers scale badly. The more bodies you have, the more constraints you have, the more work the solver has to do every frame. KSP2 is a great example of this going wrong. Every part of every vessel is running through a full constraint solve every frame, and the game runs like garbage on anything but high end hardware.

The idea I am working with is: similar physical configurations tend to produce similar outcomes. A stack of boxes falling looks basically the same whether it’s these three boxes or those three boxes. A neural net can learn to recognize those patterns and predict what the solver would have done — without actually running the solver.

So the pipeline is basically:

  1. Run the classical solver on a bunch of random scenarios
  2. Record every (state before step, state after step) pair
  3. Train a small neural net to learn that mapping
  4. At runtime, route “boring” or “familiar” physics through the net, and fall back to the real solver for anything complex or novel

The thing that makes this actually viable is that the solver generates its own training data. I don’t need a dataset. I just need to run the engine.2

Drag the slider. Find where your game breaks.

Where It’s At Right Now

The 2D engine is built in C++ using xmake — broadphase collision detection, SAT narrowphase for circles and boxes, sequential impulse resolution, the whole thing. It runs a multithreaded data generator that spins up one physics world per CPU thread, runs random scenarios, and writes binary files that Python can read for training.

The neural net itself is a small residual MLP — predicts the change in state rather than the absolute state, which keeps the outputs small and makes training way more stable. About 531k parameters, which is nothing. Runs in under a millisecond on any modern GPU.

I am still in early days — first real training run hasn’t finished yet — but the pipeline is working end to end. TensorBoard is logging everything including side-by-side videos of what the solver does vs what the net predicts, so I can actually see where it diverges.3

Why 2D First?

Honestly, 2D is just a proving ground. The goal is 3D, targeting RTX 30x0 as the minimum. But 3D introduces a bunch of problems that 2D doesn’t have — rotation alone gets way more complicated (quaternions are annoying for neural nets, there’s actually a whole paper on this4), and contact geometry becomes dramatically harder.

The 2D engine gives me a working data pipeline I can trust, training infrastructure I know works, and something to write about while the harder stuff is being figured out. Also it’s just easier to debug when you can see everything in a flat 2D view.

What’s Next

Getting the first training run done and looking at rollout stability — that’s the real metric that matters. A 1% position error per frame sounds small until you realize it compounds into garbage after about two seconds of simulation. The question is how many frames the net can run unassisted before it needs a correction from the real solver.

Drag the slider. See how fast it falls apart.

After that, the arbiter — the piece that decides which path each object takes per frame. That’s where the interesting engineering is.

3D is the end goal. Steam Deck support after that, maybe.

Footnotes

  1. Catto’s GDC papers are genuinely some of the best technical writing in gamedev. https://box2d.org/publications/↩︎

  2. This is also kind of what the paper “Learning to Simulate Complex Physics with Graph Networks” (Sanchez-Gonzalez et al., ICML 2020) does, except they use graph neural nets and I am using a plain MLP. https://arxiv.org/abs/2002.09405↩︎

  3. The video logging runs in a background thread so it doesn’t block the GPU during training. Small thing but it matters.↩︎

  4. “On the Continuity of Rotation Representations in Neural Networks” — Zhou et al., CVPR 2019. Short version: don’t use naive quaternion MSE loss, it breaks. https://arxiv.org/abs/1812.07035↩︎