I have currently a neural network to learn a (relatively non complex) system model (vector regression). Its problem is that the outputs of the system suffer from arbitry additional white gaussian noise.
I noticed that the performances of the neural network directly depend on the noise power. The results for the mse loss function had what seemed like an additional term exactly equal to the noise power (used in a system simulation for the generation of the data set). The currently used network model contains two concatenated convolutional neural networks (both contain a bias term).
My questions are:
does it make sense to use a neural network to solve such problems, especially in cases where the noise power is important?
If using a neural network remains an option (at least when the noise power is small), which techniques (layers, regulization, pooling, dropout etc..) should be able to enhance the performances of the network?