I have trained a regression model with 7 features for a given problem. Now, I have another regression problem (quite similar to the previous one) where I have only 6 samples in hand, but with 3 more features than the first model (7+3). The correlation between the target value and these 3 additional features is very high. So I would like to use transfer learning method to create a new model based on the old model; but I can not find a way to integrate the additional features into the model, since the old one was trained only on X features. On the other hand I must employ the 3 additional features in the new model which means that I cannot just remove or ignore them.



1 Answer 1


One way: Use your old model's prediction as a new feature and combine with the additional three you have, i.e. a four-length feature vector for each sample. Then, train with these new features. So, the old model will act like a static feature generator for you. Of course, you may have challenges with that low number of samples.

  • $\begingroup$ the reason of which I've turned to transfer learning is the issue of few samples of the second problem. with your proposed solution I think that I will encounter the same challenge $\endgroup$
    – jojo
    Commented Sep 24, 2021 at 19:11
  • $\begingroup$ If you are going to train & modify a transferred model, even there is no additional features, you need a data volume. Otherwise, you'll end up using the transferred model as is. e.g. I can't imagine continuing training Inception with a few images. $\endgroup$
    – gunes
    Commented Sep 24, 2021 at 19:14
  • 1
    $\begingroup$ @jojo with only 6 samples and 3 new features you are at very high risk of overfitting, getting a model that maybe works OK on those samples but doesn't generalize well. The suggestion in this answer (+1) might be the best you can do without getting more data. $\endgroup$
    – EdM
    Commented Sep 24, 2021 at 19:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.