In section 4.4.5 "Logistic regression or LDA?" of Elements of Statistical Learning by Friedman, Tibshirani and Hastie, it is claimed the following:
From the mixture formulation [that is, equation 4.38], it is clear that even observations without class labels have information about the parameters. Often it is expensive to generate class labels, but unclassified observations come cheaply. By relying on strong model assumptions, such as here, we can use both types of information.
However, I am unsure how unlabeled data could be useful in this scenario, as in that case we can't neither improve our estimation of the means, nor improve the estimation of the conditional covariance (at least directly). The only idea I could come up with to use unlabeled data is to do a (latent) Gaussian mixture model of all covariates and somehow assign labels to then fit an LDA afterwards, but the correctness of this process is not completely obvious to me. I could not find anything in the literature -besides some really sophisticated methods, which I believe were not the ideas in ESL- to do semi-supervised LDA.
I would appreciate any clarification on this matter.
