I'm training a neural network for regression using keras with about 13k training observations, each with 40 features.
It's a Sequential model with Dense layers. I generate random architectures for the hidden layers i.e. a random number of hidden layers and a random number of nodes in each layer. The input and output layers are fixed and not random.
The models are fitted and the summaries printed. The model summary tells me the number of Total params and Trainable params and Non-trainable params e.g.
Total params: 2,052,948
Trainable params: 2,052,948
Non-trainable params: 0
I am interpreting those as the weights and biases for the network.
One problem understanding this is that it tells me there are several million trainable params which is much greater than the number of observations.
The loss decreases substantially so the fitting appears to have succeeded.
How can the fitting calculate this number of parameters given far fewer available observations?