1
$\begingroup$

I'm training a neural network for regression using keras with about 13k training observations, each with 40 features.

It's a Sequential model with Dense layers. I generate random architectures for the hidden layers i.e. a random number of hidden layers and a random number of nodes in each layer. The input and output layers are fixed and not random.

The models are fitted and the summaries printed. The model summary tells me the number of Total params and Trainable params and Non-trainable params e.g.

Total params: 2,052,948

Trainable params: 2,052,948

Non-trainable params: 0

I am interpreting those as the weights and biases for the network.

One problem understanding this is that it tells me there are several million trainable params which is much greater than the number of observations.

The loss decreases substantially so the fitting appears to have succeeded.

How can the fitting calculate this number of parameters given far fewer available observations?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.