I'm learning neural networks, and wrote a network from scratch using numpy and pandas. I'm training it using stochastic gradient descent to predict house prices. The dataset is right-skewed, I mean-normalized variables other than the target, the predictor variables are of types binary and numerical. The activation function is sigmoid.

The problem I'm facing is that the output on test data is around 1 for all observations, the actual price has to be the order of 100k. I tried normalizing the target, price, in training set, then the output on test data was close to zero for all inputs.

This is how I initialized the weights and biases(sizes is a list containing number of neurons in each layer)

self.biases = [np.random.normal(0.0,b**-.5,(b,1)) for b in self.sizes[1:]]
self.weights = [np.random.normal(0.0,y**.5,(x,y)) for x,y in zip(self.sizes[1:],self.sizes[:-1])]

What should I do in order to get reasonable outputs.

Histogram of selling price of houses

  • $\begingroup$ Do you mean your network, when fitted on test data give you a regression of 1 in most cases for house prices? (I want this be true, as a buyer) How did you train the model? How could your training produce such rubbish model? $\endgroup$ – SmallChess Feb 10 '17 at 12:09
  • $\begingroup$ When fitted on the test data, did you also normalize the predictors? $\endgroup$ – SmallChess Feb 10 '17 at 12:10
  • $\begingroup$ @StudentT Yes, I did normalize the predictors for test data also $\endgroup$ – dpk Feb 10 '17 at 12:10
  • $\begingroup$ Did you look at the training error? Surely, if the target variables in the training sales are proper housing price, you can't possibly have good RMSE. $\endgroup$ – SmallChess Feb 10 '17 at 12:18
  • 1
    $\begingroup$ This might be a ridiculous comment, but did you accidentally put your output into a sigmoid function? $\endgroup$ – einar Feb 10 '17 at 12:27

The sigmoid function, pictured below, squeezes all input to fit between $0$ and $1$, as you probably know. Even a super-bad fit should probably produce some outputs above $1$ which made me suspect that you had passed your final prediction to a sigmoid function by mistake. The sigmoid function is only appreciably different from $0$ or $1$ maybe in the range $x \in [-5, 5]$.

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.