I took the online course of probabilistic graphical models (PGM) online from Daphne Koller and Eric Xing's videos, but I do not think I get the main idea of PGM, and I would appreciate it if someone can clarify some of my questions on the usefulness and methodology of PGM.

  1. What are some fields that PGM performs better than other models? I think it was the mainstream model in computer vision before deep learning took its place, and I saw many application in Daphne Koller's class materials in biological/medical domain. Is there any other field that PGM is popular?
  2. Is PGM ever useful for tabular data? I never recall PGM ever wins any Kaggle contest.
  3. Is there a general methodology of modeling using PGM? I feel I always need to write the full joint distribution of all observable/hidden variables, then use the inference/learning algorithm to get the parameters and predictions. If that is the case, then does it mean PGM can only be applied to the fields where I know pretty much everything about the all variables, so it is only useful in the fields where the prior knowledge is strong such as computer vision and biological/medical data?
  4. If PGM has to be modeled in a white box way, does that mean I cannot use it as a blackbox model to discover the patterns I cannot find by hand?
  5. What is special about the graph in PGM? I feel they only exist to allow fast identification of conditional independent variables, in order to allow fast inference/learning. In other words, the graph is like a visualization of the model. Then why do we care so much about the graph?

Here is my own answer to my questions after I read An Introduction to Conditional Random Fields by Charles Sutton and Andrew McCallum. So this only represents a very limited and naive view of PGM, and is only limited to conditional random field (CRF) which I think represents the class of model people most care about. I appreciate it if anyone can comment on my mistakes.

  1. PGM works for "structured prediction", meaning we want a multi-label output from the model. One example is named-entity recognition in NLP tasks and image segmentation of computer vision, where we want one label for each word/pixel. However, the deep learning model should already outperforms PGM in these fields.
  2. The PGM is not so useful for tabular data, e.g. in Kaggle contest, because it is rare that we want some multiple-output from such dataset.
  3. I would say from reading about CRF, PGM is extremely weak in feature discovery. CRF is basically like a logistic regression for sequence with multiple output, and we need to generate thousands or even millions of features as rules MANUALLY, features like "word A does not want to follow word B". Therefore, CRF is more like a framework that can fit the linear coefficients of this logistic-like model and has a probabilistic interpretation which is not useful most of the time. So the general working practice of doing CRF is to generate millions of rules by hand, then throw these rules and data into a CRF library.
  4. See above.
  5. The graph is totally not useful for PGM except for inferring the conditional independence between variables. The authors of the famous models usually come up with a model, then convert it into an equivalent graph, instead of the other way around. On the other hand, the conditional independence is only useful for reducing the complexity of inference/learning, and a modeler who uses off-the-shelf library does not care about the independence.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.