# Combining labeled and unlabeled data for training

When training my algorithm, if I can get some i.e. data my future test data that has no labels can it improve my algorithm's efficiency, is there any mathematical proof for it?

PS: I think semi-supervised methods and transductive learning are something similar to what I look for.

-

Yes unlabelled data can improve performance, and the methods used are normally described as "semi-supervised" or "transductive" learning. Without knowing more about the application is isn't really possible to suggest suitable algorithms, but there is a very good book edited by Chapelle, Scholkopf and Zien, called "Semi-supervised learning" (MIT Press) which would be a good place to start for semi-supervised learning.

Mathematical proof is somewhat difficult as it will help in some situations, but not others, so the best you can do is bounds on generalisation performance. Chapter 8 of Vladimir Vapnik's book "Statistical Learning Theory" is probably what you want on transductive learning.

-
Does the term of semi-supervised learning and transductive learning same? Also can you provide any paper for such kind of explanation? – kamaci Jan 12 '13 at 17:11
No, they are not quite the same thing. In transductive learning, the unlabelled points are the test points where you actually want to make predictions (and the output of the algorithm may be just the predictions for those points, rather than a model that can be used to make predictions anywhere). For semi-supervised learning, the unlabelled points don't have to be test points and the output of the algorithm is a predictive model. Vapnik's book is probably the best source for justifications of the transductive approach. – Dikran Marsupial Jan 12 '13 at 17:21
What you mean with predictive model? – kamaci Jan 12 '13 at 17:39
I just mean a model that you can use to make predictions (for instance for a linear regression problem, a semi-supervised method would give you the regression coefficients of the model, but a transductive method might just tell you the predicted values for all of the test points, but wouldn't give you the regression coefficients). – Dikran Marsupial Jan 12 '13 at 17:57

The paper Ira Cohen et. al., "Semisupervised Learning of Classifiers: Theory, Algorithms, and Their Application to Human-Computer Interaction" contains the results you are looking for.