Gaussian discriminant analysis models learn $P(x|y)$ and then apply Bayes rule to evaluate $$P(y|x) = \frac{P(x|y)P_{prior}(y)}{\Sigma_{g \in Y} P(x|g) P_{prior}(g) }.$$ Hence, they are generative models. Why then is it called discriminant analysis? If it is because we finally derive a discriminant curve between the classes, then that happens for all generative models.
$\begingroup$
$\endgroup$
asked Sep 1, 2011 at 18:06
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
If you mean LDA I would say the name, linear discriminant analysis, can be explained historically dating back at least to Fisher's paper from 1936, which, to the best of my knowledge, precedes the current terminology and distinction in machine learning between a discriminative and a generative model. Not that Fisher called it linear discriminant analysis directly, but he did explicitly ask for a linear function for discrimination. As a curious side remark, Fisher considered discrimination for the famous Iris data set in the paper.
Fisher did, by the way, not present the linear method for discrimination in terms of a generative model. He sought a linear combination (for two classes) that maximizes the ratio of the between-group variance to the within-group variance, which does not require a normality assumption. Details, and how it relates to LDA as a Bayes rule for a generative model, can be found in Chapter 3 in Brian Ripley's book "Pattern Recognition and Neural Networks".
$\begingroup$
$\endgroup$
It is simple, in case you have two classes $(Y=0 , Y=1)$, the GDA makes use of this assumption:
- $P(X|Y=0) \sim \mathcal{N}(\mu_0,\Sigma_0) $
- $P(X|Y=1) \sim \mathcal{N}(\mu_1,\Sigma_1)$
- $P(Y=1)=1-P(Y=0)=\Phi$
And then gets the parameters $(\mu_0,\Sigma_0,\mu_1,\Sigma_1,\Phi)$ using maximum likelihood estimation.
So it's Gaussian because it uses a gaussian assumption for the intra-goup distribution (you may want to use uniform instead for ex) and discriminant because it aims to separate data into groups.
You can find more info here.
