Differentiable Simulators

Why Differentiable?

Gradients can be computed natively.
- policy gradients for control tasks
- dynamics jacobians for model-based control
- physical property gradients for model fitting
- other cool things

300 300

Alternatives:
- Finite differencing: evaluating the forward simulation multiple times with small perturbations to the relevant parameters to approximate the relevant gradients
- Gradient-free: inefficient and approximate

the high computational burden of finite differencing when computing the gradient with respect to a large number of model/policy parameters
the instability of numerical gradients over long time horizons, especially if contacts change over the course of a rollout (roundoff errors)

Existing simulators simply add an analytical description of the LCP problem. Typically utilizing a library like pytorch that enables autograd.

from Filipe de A. Belbute-Peres ... Tenenbaum/Kolter

First, we show that it can infer the mass of an object by observing the dynamics of a scene.
Next, we demonstrate that embedding a differentiable physics engine within a deep autoencoder network can lead to high accuracy predictions and improved sample efficiency.
Finally, we use the differentiable physics engine together with gradient-based control methods to show that we can learn to perform physics-based tasks with low sample complexity when compared to model-free methods.

Dojo
Nimble
...
eventually Mujoco
IsaacSim