1
$\begingroup$

Background

I have a time-series prediction model which aggregates historic analogues to make predictions about potential future positions. It works really well in that it makes accurate predictions and is driven by strong business intuitions. However, making predictions is prohibitively calculation intensive (it compares all possible potential states of the system to all historically observed states at each iteration) to the point that it is infeasible to implement in a live system.

Proposal

To combat this issue, my plan is to implement a neural network to predict the results of my current model, using predictions from the old model to augment the historic dataset with what-if scenarios in order to increase the size and variability of the training set. (I have previously tried solving the initial prediction problem using neural networks but found it hard to get good results without overfitting the training data due to the small number of, and lack of variation in, the historic observations so am hoping that augmenting the dataset in this way will overcome these issues)

Are there any intrinsic problems with this approach? I foresee a certain amount of propagating and compounding errors, but am more interested in whether there are strong reasons against using models to predict models in general.

$\endgroup$
1
$\begingroup$

This is a valid approach. Neural networks are indeed often used to simulate results of some exact, computationally heavy models. For example, Nvidia speeds up raytracing using neural nets. The benefit is, since your model is very accurate, it can generate a huge dataset, which is exactly what neural networks need.

Sometimes, this is also successfully used in the form where a large neural network is trained, and then a second, smaller one is trained to simulate its outputs, leading to a compact, computationally more efficient model with similar performance. Relevant articles on the topic include Do Deep Nets Really Need to be Deep? and Distilling the Knowledge in a Neural Network.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.