0
$\begingroup$

I was working with keras and tensorflow as backend on an NLP problem when I observed that increasing my training data size caused an increase in the number of trainable parameters even when batch size remained the same. From what I understand, trainable parameters are the weights which are learnt for each layer. If that is the case then it should not change irrespective of whether I increase or decrease my input data size.

So what is exactly happening here? The reason why this is important is because I perform normalization upon my data once it is fully loaded. This normalization would not work properly if I used a generator function.

$\endgroup$
0
$\begingroup$

I just realized that increasing training data leads to an increase in the number of tokens for training when embedding layers are set to trainable. This causes an increase in the number of trainable parameters.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.