According to this link LDA is a generative classifier. But the name itself has got the word 'discriminant'. Also, the motto of LDA is to model a discriminant function to classify. Then why is this a generative model?
$\begingroup$
$\endgroup$
3
-
$\begingroup$ LDA is a generative model because it uses the joint probability distribution, P(x,y). $\endgroup$– Alexander BeatsonCommented Aug 5, 2019 at 9:48
-
$\begingroup$ Welcome to Stats.SE. Can you please expand you answer and provide further insight? It will help both the original poster and other people with a similar questions. By the way, take the opportunity to take the tour( stats.stackexchange.com/tour), if you haven't done it already. See also some tips on formatting help and on writing down equations using LaTeX / MathJax. $\endgroup$– Ertxiem - reinstate MonicaCommented Aug 5, 2019 at 10:27
-
$\begingroup$ That's Latent Dirchlet Allocation and not Linear Discriminant Analysis. $\endgroup$– Farhood ETCommented Jul 10, 2021 at 9:47
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Classification is the problem of assigning data samples $x \in X$ to classes $k \in G$. To solve classification, we need to model the probability of class conditional on data, $P(G=k \mid X=x)$. Discriminative models do just that, directly.
Generative models model the joint probability of data and class $P(G=k, X=x)$.
How are joints and conditionals related? By the equality $P(G=k, X=x) = P(G=k \mid X=x) P(X=x)$
In other words, the joint probability of data and class modeled by generative models, $P(G=k, X=x)$, can be factored into, on one hand, the probability of class conditional on data, $P(G=k \mid X=x)$, and a data model $P(X=x)$ on the other hand. The first factor is necessary and sufficient to solve the classification problem. The second factor, the data model, can be used to generate new samples (thus the name.)
By drawing samples from the data model, one is in fact producing synthetic data, or new data points with similar statistics to the original data used to fit the generative model. How similar the synthetic and real data are will depend on how well the model captures the data distribution$-$but that is beyond the scope of the question.
Why does LDA bother with data modeling when its aim is classification (discrimination)? Because it takes into account data structure (specifically, data covariance in addition to cluster means) when computing the decision rule, and does so in a way that a data model is obtained for free (since the data model is Gaussian, mean and covariance is all it takes).
