I feel that most generative models happen to be DGM(directed graphical model), and most discriminative models are UGM(undirected graphical model). Is there any correlation between these concepts?
- generative model and DGM, e.g. Navie Bayes/Mixture Gaussian/HMM
- discriminative model and UGM, e.g. Logistic regression/CRFs
- generative model and UGM: ??? Ising Model ($p(x,y)=\frac{exp(-E(x,y))}{Z}$)
- discriminative and DGM: any examples?
In generative model(or DGM), we model the joint probability $P(X,Y|\theta)$($\theta$ is hyperparameter, $X$ is hidden variable). The objective is to maximize the marginals $P(Y|\theta)$ w.r.t $\theta$. Because we specify the joint distribution, we can make inference about $X$.
For discriminative model(or UGM), we specify $P(Y|X,\theta)$ and try to maximize it directly. e.g. Logistic regression, $logit[P(Y_n=1|x_n)]=\beta x_n$