Issues with implementing neural network

Question

I am trying to use neural network to learn a non linear function mapping input to outputs. However, I am having some issues with it. I used tansig activation function for the hidden layers and for the output I used logsig. I scaled the output variables in the range [0 1]. The input variables were standardized to zero mean and unit std. I learn the neural network. Now when I test my NN model on either the train set or a different test set, the outputs are always greater than 0.5. Why is it so? My targets could be anything from 0 to 1. However, my outputs from the model are at least 0.5. Any suggestions what could have happened?

I have around 600 features with 5000 training examples and 10 neurons in the hidden layer and a single output

I tried with tansig activation function for the hidden layers and a linear activation function for the output layers and I got the following. This is somewhat better than before. However, I am a bit surprised why bias is introduced

Answer 1

I can see three potential reasons for that:

• $logsig$ values abouve $0.5$ simply mean, that sum of activations from the hidden layer is always non-negative. Maybe you forgot to include the bias (bias neuron) in your hidden layer?
• You standarized your data using some mapping $\phi(x_i)\rightarrow x'_i$. Maybe you trained your network with $(x'_i, y_i)$ and tested on $(x_i,y_i)$ (forgetting about input standarization)?
• You mapped your outputs to $[0,1]$ and network learned values in $[0.5,1]$, maybe there is some implementation error and you actually provided network with training set of form $(x'_i, logsig(y_i))$ instead of $(x'_i,y_i)$?