1
$\begingroup$

I have been working on a neural network based predictor for a project. The aim is to learn a certain quantity, say the signal strength of a cellular network, for each coordinate set in the dataset. There is a discernible pattern to the data which I know from plotting it i.e. high signal strength near a tower and low further away. However, the output of the network seems to struggle to go beyond a certain maximum and minimum value.

Here are some details of what I have tried to improve this and my overall design:

  • I am using a two layer feed forward fully connected NN.
  • The implementation is in Keras using a tensorflow backend.
  • The input coordinates have been rescaled to between 0 and 1 and I have tried a cartesian input conversion too.
  • The two layers have sigmoid activations and the ouput layer is linear.
  • I have trained the network extensively, up to 10k epochs and the results are pretty much the same.
  • There is literature already showing that modeling signal strength is possible with this set of inputs

In my experience, an output limiting itself is due to the activation being oversaturated, often due to the input not being scaled. However, I think I have catered to that reasonably well. Changing the number of neurons, additional layers, and even activations (relu, linear, tanh) didn't help. And I am using the Adam optimizer with default values.

Improve this question
$\endgroup$
4
  • $\begingroup$ how about rescaling the output to a reasonable range? $\endgroup$
    – shimao
    Commented Aug 20, 2018 at 10:17
  • $\begingroup$ I tried that too. It has no effect on the results $\endgroup$
    – malesam
    Commented Aug 20, 2018 at 12:53
  • $\begingroup$ What kind of loss function are you using? If you are using a mean square error loss for example, you neural network might have learned that it is risky (i.e. very large loss) to predict very large or very small values as it hasn't found a good feature to predict them accurately. $\endgroup$
    – GR4
    Commented Aug 24, 2018 at 13:53
  • $\begingroup$ I have tried a bunch of loss functions, and mean absolute error tends to work the best. And yes, this behavior was made worse using mean square error. $\endgroup$
    – malesam
    Commented Aug 25, 2018 at 1:39

1 Answer 1

Reset to default
0
$\begingroup$

I ended up rephrasing the entire problem as a time series prediction modeled with a recurrent NN with LSTM cells. This yielded significantly better results. I think that the problem as I was phrasing it originally was beyond the learning abilities of a simple feed-forward neural network and it was trying to converge to the mean

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.