# Difference between GMM classification and QDA

I know that every class has the same covariance matrix $\Sigma$ in linear discriminant analysis (LDA), and in quadratic discriminant analysis (QDA) they are different. When using gaussian mixture model (GMM) in supervised classification, we fit a Gaussian with its own mean and variance to each class in data. So what is the difference between QDA and GMM?

Is my understanding of GMM wrong? Maybe I should fit more than one Gaussian to each class in order to model subgroups in it. But I am not sure if this is true or not.

• Please don't forget, when asking a question, to expand your abbreviations. – ttnphns Mar 1 '16 at 8:03
• A discriminant analysis is not only a classification but also a dimensionality reduction. In DA, classifiers are the discriminants extracted, not the input variables. – ttnphns Mar 1 '16 at 8:06
• So, if we consider just the classification part, are QDA and GMM the same? – groove Mar 1 '16 at 9:04

If you're given class labels $$c$$ and fit a generative model $$p(x, c) = p(c) p(x|c)$$, and use the conditional distribution $$p(c|x)$$ for classification, then yes you're essentially performing QDA (the decision boundary will be quadratic in $$x$$). Under this generative model, the marginal distribution of the data $$x$$ is exactly the GMM density (say you have $$K$$ classes):
$$p(x) = \sum_{k \in \{1,...,K\}} p(c=k) p(x|c=k) = \sum_{k=1}^K \pi_k \mathcal{N}({x};{\mu}_k, {\Sigma}_k)$$
"Gaussian mixture" typically refers to the marginal distribution above, which is a distribution over $$x$$ alone, as we often don't have access to the class labels $$c$$.