I aim for replicating an numerical (non stochastic) algorithm by a neural network. Therefore I have basically an unlimited amount of data and I wish that the network have an almost perfect fit in terms of MAPE.That means I wish:
- MAPE should be around 0.5 %
- Maximum absolute procentual deviation should not exceed 5%
The problem is that the algorithm outputs quite small values (of the order of 10^-4) and therefore minimizing the L2 or L1 error directly on them (or a scaled version) does not translate to good MAPE performance. Choosing MAPE as training loss didn't work well (as expected). I then logarithmized the values which helped a lot but MAPE is still around 4% and is unevenly distributed (A subset of smaller values have bigger MAPE). I find that unsatisfying and have the following questions:
- Are neural nets the appropriate method here? I have the feeling that neural nets just are not able to approximate a deterministic function without noise with high precision.
- Is there anything else I can do to get a better MAPE Performance?
- I had the idea to increase frequency of data samples on which MAPE is high. It seems to me that there should be literature on it but I lack the right search term. It reminds a little of active learning but I could't find relevant literature. Could you provide something like that or tell my why the idea is bad?
I am not sure if it is relevant but my architecture consisted of modules of the type [Linear, Relu, Batch_norm] with skip connections. I experimented with various depths and widths but I couldn't decrease the MAPE below 4% (on the training set). The data consists of 23 explanatory variables and the size of my (generated) dataset is 3 million.