# What is the relationship between discriminative and generative and directed and undirected graphical models?

I feel that most generative models happen to be DGM(directed graphical model), and most discriminative models are UGM(undirected graphical model). Is there any correlation between these concepts?

• generative model and DGM, e.g. Navie Bayes/Mixture Gaussian/HMM
• discriminative model and UGM, e.g. Logistic regression/CRFs
• generative model and UGM: ??? Ising Model ($$p(x,y)=\frac{exp(-E(x,y))}{Z}$$)
• discriminative and DGM: any examples?

In generative model(or DGM), we model the joint probability $$P(X,Y|\theta)$$($$\theta$$ is hyperparameter, $$X$$ is hidden variable). The objective is to maximize the marginals $$P(Y|\theta)$$ w.r.t $$\theta$$. Because we specify the joint distribution, we can make inference about $$X$$.

For discriminative model(or UGM), we specify $$P(Y|X,\theta)$$ and try to maximize it directly. e.g. Logistic regression, $$logit[P(Y_n=1|x_n)]=\beta x_n$$

• I found the example of discriminative DGM, maximum-entropy Markov model (MEMM). My feeling is that most DGM are generative. – H.Yuanchen Apr 18 '19 at 15:52

Directed Graphical Models, or Bayesian Networks, model the joint distribution with a specific factorization. Thus, when using DGM, you always consider the step of writing the joint law $$p(X,Y)$$ from which you can deduce $$p(X)$$ and $$p(Y|X)$$ and what you get is a generative model (with $$X$$ being the hidden variable and $$Y$$ the observed variable).
On the other hand, when using an Undirected Graphical Model, you must consider writing potential functions which a are much less intuitive and harder to use than conditional probabilities used in DGM. That may be why you are feeling this tendency of UGM to be discriminative : energy functions are hard to write, "don't waste ressources" to model the full $$p(X,Y)$$ if $$p(X|Y)$$ is all that interests you. (Quote from Machine Learning, Chapter 19, Kevin P; Murphy)