I have data set with 200 features and I am running it through nn with 3 hidden layers.

I get 0 loss on training data set, but here comes the interesting part.

If I choose to select only 100 features of the same data set and use the hyper parameters (same loss function, optimizer etc) I can not converge loss function bellow 0.3. How could this be explained?


The data drives the parameters of the neural networks towards a task.

So I think, when you alter your dataset, the solution in parameter space is converged to a distinctively. In your case, you had 200 features originally, that would have perfectly captured the variation in the data. But, when you selected only 100 features, without checking their importance in the dataset, your model would not perform better. The loss values indicate the same in your case.

I would suggest, trying PCA and get first 100 features ranked by the variance explained. In that case, you may see a better loss than 0.3. But it could still be more than zero. Assuming all other network settings are the same.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.