I have been working on a neural network based predictor for a project. The aim is to learn a certain quantity, say the signal strength of a cellular network, for each coordinate set in the dataset. There is a discernible pattern to the data which I know from plotting it i.e. high signal strength near a tower and low further away. However, the output of the network seems to struggle to go beyond a certain maximum and minimum value.
Here are some details of what I have tried to improve this and my overall design:
- I am using a two layer feed forward fully connected NN.
- The implementation is in Keras using a tensorflow backend.
- The input coordinates have been rescaled to between 0 and 1 and I have tried a cartesian input conversion too.
- The two layers have sigmoid activations and the ouput layer is linear.
- I have trained the network extensively, up to 10k epochs and the results are pretty much the same.
- There is literature already showing that modeling signal strength is possible with this set of inputs
In my experience, an output limiting itself is due to the activation being oversaturated, often due to the input not being scaled. However, I think I have catered to that reasonably well. Changing the number of neurons, additional layers, and even activations (relu, linear, tanh) didn't help. And I am using the Adam optimizer with default values.