1
$\begingroup$

If I fit my model on the same training set twice or thrice so does the model remember the learning from each iteration and improve ?

$\endgroup$

1 Answer 1

1
$\begingroup$

TECHNICALLY, IT IS POSSIBLE, BUT SUCH A DESCRIPTION IS A STRETCH.

It depends on the type of model and, critically, how it is trained.

For a neural network, you could start again from where the network left off, though this would be equivalent to using more training epochs. If, for whatever reason, you want implement this in software, you would have to save the network weights and use them as the starting point when you train the next time.

For a linear regression, the (standard) solution is always given by a matrix product: $\hat\beta_{ols}=(X^TX)^{-1}X^Ty$. Unless you apply a numerical method to solve for this and it takes a long time to solve, whenever you run this calculation, you will get the same result every time.

Overall, if you have to do a process over and over (e.g., gradient descent), picking up where you left off could be a way to improve the (in-sample) fit, though this is likely already occurring under the hood.

$\endgroup$
1
  • $\begingroup$ One way of interpreting OP's question might be that one duplicates the training data. You're right that the estimates for OLS would be the same, but the uncertainty would not. $\endgroup$ Oct 29, 2022 at 14:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.