I have a neural network model with 20 layers. There are 30 input nodes and 5 output nodes. I am using backpropagation algorithm to train the model. I can see that in the first 5 layers, the weights are not changing. I don't understand why that is happening. Can someone point me in the direction of how to fix it?
There is nothing to fix. All knowledge you put into your model can be memorized in those last 15 layers. Probably your model is totally over fitting. Explanation: Back propagation works from layer to layer and tries to adjust weights to achieve expected results. If you have narrow and deep model expected result can be found just after altering 15 layers even if you have 20. It also can easily cause model over fitting that happened to you. You have high accuracy with training set, and high error with validation set. To avoid it you need to acquire much more data that will be able to "fulfill" model, or shrunk model to cause it generalize it's training.