I am reading Caruana (1997) Multitask learning (pdf). In the definition of multi task learning, the author states that;

Usually, we do not care how well extra tasks are learned; their sole purpose is to help the main task be learned better.

This is also the goal of transfer learning right? What is the difference then? Are these two terms used interchangeably (I think not)?

  • 1
    $\begingroup$ There's a lot of confusion around the exact meaning of many terms related to transfer learning -- partly because people are using it in many applications which differ slightly in format. $\endgroup$ Commented Dec 10, 2019 at 14:41

3 Answers 3


Single Task Learning: Giving a set of learning tasks, t1 , t2 , …, t(n), learn each task independently. This is the most commonly used machine learning paradigm in practice.

Multitask Learning: Giving a set of learning tasks, t1 , t2 , …, t(n), co-learn all tasks simultaneously. In other words, the learner optimizes the learning/performance across all of the n tasks through some shared knowledge. This may also be called batch multitask learning. Online multitask learning is more like lifelong learning (see below).

Transfer Learning (or Domain Adaptation): Giving a set of source domains/tasks t1, t2, …, t(n-1) and the target domain/task t(n), the goal is to learn well for t(n) by transferring some shared knowledge from t1, t2, …, t(n-1) to t(n). Although this definition is quite general, almost the entire literature on transfer learning is about supervised transfer learning and the number of source domains is only one (i.e., n=2). It also assumes that there are labeled training data for the source domain and few or no labeled examples in the target domain/task, but there are a large amount of unlabeled data in t(n). Note that the goal of transfer learning is to learn well only for the target task. Learning of the source task(s) is irrelevant.

Lifelong Learning: The learner has performed learning on a sequence of tasks, from t1 to t(n-1). When faced with the nth task, it uses the relevant knowledge gained in the past n-1 tasks to help learning for the nth task. Based on this definition, lifelong learning is similar to the general transfer learning involving multiple source domains or tasks. However, some researchers have a narrower definition of lifelong learning. They regard it as the learning process that aims to learn well on the future task t(n) without seeing any future task data so far. This means that the system should generate some prior knowledge from the past observed tasks to help new/future task learning without observing any information from the future task t(n). The future task learning simply uses the knowledge. This definition makes lifelong learning different from both transfer learning and multitask learning. It is different from transfer learning because transfer learning identifies prior knowledge using the target/future task labeled and unlabeled data. It is different from multitask learning because lifelong learning does not jointly optimize the learning of the other tasks, which multitask learning does.

All content lifted from https://www.cs.uic.edu/~liub/IJCAI15-tutorial.html.

  • $\begingroup$ Since this is just a wholesale extract from another site, it does not contribute anything. Would you like to include some explanation? $\endgroup$
    – whuber
    Commented Jan 12, 2017 at 21:47
  • $\begingroup$ @whuber sure, i will explain some intuition behind them a bit later. $\endgroup$
    – Rafael
    Commented Jan 13, 2017 at 23:58
  • $\begingroup$ This definition of transfer learning does not sound so good today. It is often claimed that "words embeddings" are the method of choice for transfer learning in Natural Language Processing. In this case, the source task has many unlabeled data, and no labeled data; only the target task has labeled data. Furthermore, the target task is not know when training words embeddings on the source task. $\endgroup$
    – xtof54
    Commented Dec 10, 2019 at 14:19
  • $\begingroup$ Can we describe transfer learning as a transfer of knowledge between models, while in multi-task learning the transfer happens within the model? Hence the proximity between both and the similar problematics such as negative transfer. $\endgroup$
    – Eric Burel
    Commented Feb 24, 2023 at 8:39

Multi-task Learning and Transfer Learning methods although they have some things in common, they are not the same. Transfer Learning only aims at achieving high performance in the target task by transferring knowledge from the source task, while Multi-task Learning tries to learn the target and the source task simultaneously.

A major assumption in many machine learning and data mining algorithms is that the training and future data must be in the same feature space and have the same distribution. This assumption is weak and in many cases may not hold. For example, imagine that we have a classification task in one domain of interest, but we only have sufficient training data in another domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In Multi-task Learning there is no mandatory to observe a difference in the distributions of the different tasks.

Transfer Learning is a broader topic than Domain Adaptation. The latter is the process that attempts to alter the source domain in a way to bring the distribution of the source closer to that of the target domain. Transfer Learning can include also the case when we want to transfer knowledge from an old task to newer and different one (e.g. in a classification problem that we have different labels), which has some similarities with the old, apart from the above explained situation where we have one task but there is a difference in the distributions of the data (e.g. sample bias etc).


There is a slightly different (and I think better) answer in the ICML 2018 review ( https://towardsdatascience.com/icml-2018-advances-in-transfer-multitask-and-semi-supervised-learning-2a15ef7208ec ), where multi-task learning, domain adaptation and fine-tuning are specific cases of transfer learning.

Lifelong learning (or Metalearning) accumulates knowledge from multiple "historical" tasks seen in the past, with the objective of improving performances on future tasks. It's related to multi-task learning, except that the tasks come from the past, and the targets are in the future.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.