1
$\begingroup$

Say we have n similar data sets, with the same variables, and outcome labels x and y. In these data sets, domains slightly differ as suggested by the proportion of the minority class x (ranging from 1%-15%).

  • How can we develop a robust ML algorithm that works out-of-the-box on new data sets with varying proportions of the classes (i.e. 1-99 or 15-85)?
  • Which algorithms are a good-fit for this purpose?
  • And (how) is this related to transfer learning, domain adaptation and multi-task learning?
$\endgroup$
1
$\begingroup$

First-off: I don't think that there is just one truth about your questions.

And (how) is this related to transfer learning, domain adaptation and multi-task learning?

Transfer learning is exactly what you describe. In your case, the problem domain differs, but the structure remains the same. The research area of transfer learning in general focuses on three questions: when, how and what to transfer. Overview: https://link.springer.com/article/10.1007/s10462-013-9406-y

How can we develop a robust ML algorithm that works out-of-the-box on new data sets with varying proportions of the classes (i.e. 1-99 or 15-85)? Which algorithms are a good-fit for this purpose?

Generally, if you apply ML, you strive to build models that represent concepts about your target, or "general knowledge". In your case, the goal is to not build models that "remember data", meaning, that overfit. If you try to predict the same thing on both data sets A and B: just build a good model on data set A and it should work properly on B. Problem: if your data is for instance a set of blog entries and you want to determine the sentiment, the domain might as well matter a lot. If A is about movies and B is about politics, general concepts about sentiments might be the same, but domain-specific "signaling" words might differ in their sentiment. Transferring a model without adjustment might therefore not be a good choice. Thus, starting of with a transferred model and adapt it somehow to the new data might work!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.