I'm currently working on a Dueling-Double DQN model, and I noticed that though the loss (mse of Q values between training and target networks) seems to be decreasing, the distribution of weights in the network almost didn't change, while the biases varies a lot during training, resulting in Q values update.
hidden layers: 2
input nodes(state space): 80
hidden nodes: 256
output nodes(action space): 3
Since I'm using relu as activation function so I suppose gradient vanishing is not the reason, what other reason could cause the behaviour? Is there any direction for hyperparameter tuning since grid search costs a lot of time. Thanks.