What are some instructive examples of machine learning reproducing the behaviour of simulators?

Question

I have a computer model which reproduce the behaviour of a physical system we want to control. The model includes a bit of fluid dynamics, heat exchange, pressure calculations etc. Inputs include both prior information like pipe geometry and real-time information like how we choose to control pumps.

I would like to build a machine learning system that "learns" from a large number of model-runs for the following purposes:

1. A "stand-in" for the orignal model, that produces approximate results a lot faster. (similar to what caching and interpolation methods may achieve)
2. To later learn from real-world system runs, so as to transcend the original simulator in some aspects. (e.g "learn" the correct heat-loss from a pipe)
3. Eventually, learn to control the system more optimally

Starting out with purpose 1 I'm at a loss for good practical examples. I can see that both reinforcement learning and system identification are relevant. Also, deep learning might be applicable, given that I can produce an unlimited number of simulator runs. However, case studies that could help single out practical and relevant approaches would be most appreciated.

Answer 1

World Models by Ha and Schmidhuber presents a pretty relevant example of such a system. The main components are

1. A learned latent representation of the world state -- it's easier to reason about enemies or walls in an environment rather than a bunch of pixels. Although not necessary if you already have a good state representation.
2. A dynamics model to predict how the world state will evolve over time and in response to some action. In other words, this is the simulator.
3. A training algorithm and policy network which can be trained on this learned dynamics model.
4. A procedure for alternating between improving the simulator and training on the simulator.

This allows for very low sample complexity (in terms of number of rollouts from the real environment you have to perform)