8
$\begingroup$

I have already trained a neural network on my data. In the future, I will receive some more data. How can I incorporate this data into my model without rebuilding it from scratch?

$\endgroup$
  • 1
    $\begingroup$ I don't know about NNets, but Bayesian models can updated using new data, without rebuilding from scratch and without loosing the information they learned from the previous data. $\endgroup$ – Skander H. Jun 22 '18 at 23:09
11
$\begingroup$

In keras, you can save your model using model.save and then load that model using model.load. If you call .fit again on the model that you've loaded, it will continue training from the save point and will not restart from scratch. Each time you call .fit, keras will continue training on the model. .fit does not reset model weights.

I would like to point out one issue that might arise from training your model this way though, and that issue is catastrophic forgetting. If you feed your model examples that differ significantly from previous training examples, it might be prone to catastrophic forgetting. This is basically when the neural network learns your new examples well and forgets all the previous examples because you are no longer feeding those examples to it. It arises because as optimizers get more efficient, the neural network will be get more efficient at fitting new data quickly - and the best way to fit the new data quickly might be to forget old data. If your future data is very similar to your current data, then this won't be a problem. But imagine you trained a named entity recognition system to recognize organizations. If in the future you feed it a bunch of data that teach it how to recognize people names, it might catastrophically forget how to recognize organizations.

$\endgroup$
0
$\begingroup$

The model will continue to learn each time you call model.fit.

You can save a model using model.save and load it with load_model.

Here is a simple example:

from keras.layers import SimpleRNN, TimeDistributed

model=Sequential()
model.add(SimpleRNN(input_shape=(None, 2),
                    return_sequences=True, 
                    units=5))
model.add(TimeDistributed(Dense(activation='sigmoid', units=3)))
model.compile(loss = 'mse', optimizer = 'rmsprop')
model.fit(inputs, outputs, epochs = 500, batch_size = 32)

model.save('my_model.h5')

from keras.models import load_model
model = load_model('my_model.h5')

# continue fitting
model.fit(inputs, outputs, epochs = 500, batch_size = 32)
$\endgroup$
  • $\begingroup$ It's possible to just train the model on the new data. But what assumptions does this make? Is this different than training a new model on all of the data, new and old together? Does the model that's trained for one epoch on the new data have the same predictions as the model trained with new and old data together? Why or why not? $\endgroup$ – Sycorax says Reinstate Monica Jun 22 '18 at 19:39
  • $\begingroup$ when you call fit the second time is the model building upon what was previously learned? Or, is it starting over and learning from scratch? $\endgroup$ – yalpsid eman Jun 22 '18 at 20:41
  • $\begingroup$ see here: github.com/keras-team/keras/issues/4446 $\endgroup$ – ahstat Jun 22 '18 at 21:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.