# What is difference between 'transfer learning' and 'domain adaptation'?

Is there any difference between 'transfer learning' and 'domain adaptation'?

I don't know about context, but my understanding is that we have some dataset 1 and train on it, after which we have another dataset 2 for which we want to adapt our model without retraining from scratch, for which 'transfer learning' and 'domain adaptation' help solve this problem.

According to the field of Convolutional Neural Networks:

• By 'transfer learning' I mean 'finetuning' [1]

• In this case [2] it's unsupervised, but should 'domain adaptation' always be unsupervised?

• A Survey on Transfer Learning by Pan et. al, 2009 provides some insight. They categorize domain adaptation as a type of transfer learning. Mar 29, 2017 at 22:53

It seems that there are some disagreement between researchers on what the difference between 'transfer learning' and 'domain adaptation' is.

From {0}:

The notion of domain adaptation is closely related to transfer learning. Transfer learning is a general term that refers to a class of machine learning problems that involve different tasks or domains. In the literature, there isn't yet a standard definition of transfer learning. In some papers it's interchangeable with domain adaptation.

From {1}:

References:

From Hal Daume's article [1]:

The standard classification setting is a input distribution p(X) and a label distribution p(Y|X). Domain adaptation: when p(X) changes between training and test. Transfer learning: when p(Y|X) changes between training and test.

In other words, in DA the input distribution changes but the labels remain the same; in TL, the input distributions stays the same, but the labels change.

• Welcome to Cross Validated! Please take a moment to view our tour. It is preferred that answers are self-contained with proper citations. If you were to use the information from your link to provide a reasonable answer to the question with the link for further reading, this would be a much improved answer. Mar 13, 2017 at 19:31
• I don't believe this explanation. I don't believe HAL here. What if both $p(Y|X)$ and $p(X)$ change at the same time? Apr 27, 2017 at 16:19
• I agree with @caveman. For many transfer learning scenarios that's the case.
– pir
Sep 9, 2017 at 21:27

Throughout the literature on transfer learning, there is a number of terminology inconsistencies. Phrases such as transfer learning and domain adaptation are used to refer to similar processes. Domain adaptation is the process of adapting one or more source domains for the means of transferring information to improve the performance of a target learner. The domain adaptation process attempts to alter a source domain in an attempt to bring the distribution of the source closer to that of the target. In the Domain Adaptation setting the source and target domains have different marginal distributions p(X). According to Pan's survey, Transfer Learning is a broader term that can also include the case when there is a difference in the conditional distributions p(Y|X) of the source and target domains. In contrast, Daume discriminates the two terms [1], referring that Domain Adaptation is when p(X) changes between source and target and Transfer learning is when p(Y|X) changes between source and target domains.

• Would it be possible to add a citation for "Pan's survey"? I think it would make this answer more useful to future readers. Jul 16, 2017 at 21:31
• Pan, Sinno Jialin, and Qiang Yang. "A survey on transfer learning." IEEE Transactions on knowledge and data engineering 22, no. 10 (2010): 1345-1359. scholar.google.com/… ; citeseerx.ist.psu.edu/viewdoc/… Jul 16, 2017 at 21:33

I think that "Transfer Learning" is a more general term, and "Domain Adaptation" is a scenario of "Transfer Learning".

[1] Transferable Attention for Domain Adaptation. http://ise.thss.tsinghua.edu.cn/~mlong/doc/transferable-attention-aaai19.pdf

According to [1], domain adaptation is the transfer learning in NLP: "Transfer learning in the NLP domain is sometimes referred to as domain adaptation."

[1] Pan, S. J., and Q. Yang. “A Survey on Transfer Learning.” IEEE Transactions on Knowledge and Data Engineering 22, no. 10 (October 2010): 1345–59. https://doi.org/10.1109/TKDE.2009.191 or https://www.cse.ust.hk/~qyang/Docs/2009/tkde_transfer_learning.pdf

It seems wikipedia has the most concise answer:

Domain adaptation is a subcategory of transfer learning. In domain adaptation, the source and target domains all have the same feature space (but different distributions); in contrast, transfer learning includes cases where the target domain's feature space is different from the source feature space or spaces.

Here is the source.

• While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review May 12, 2022 at 13:51