Maximum Likelihood Estimation for Conditional Random Field parameters

Question

I have a custom potential function for a Conditional Random Field (CRF) very similar to Fei Fei Li's work. In this work, the parameter learning is done by Maximum Likelihood Estimation. I would like to ask if there are any MLE solvers for general CRFs ?

Similar to this work, my three nodes stands for human pose, action and object. And unfortunately, I could not find any CRF implementation which could account for non-homogeneity in the feature space.

I had a brief conversation with Andreas from PyStruct (link skipped due to lack of reputation) and currently it does not support non-homogeneous nodes. For my code to work, it has to be all action, all hands or all object features.

Also, I went through the documentation provided in UGM and went through the TrainMRF and TrainCRF implementation in details. The same issue persists there as I found that all the nodes were homogeneous and hence all features are from the same feature space.

I tried an odd example where I concatenated all my features and used the GraphCRF in PyStruct. As expected, it works, but at the expense of the lost CRF structure.

So, (1) Can you please suggest MLE solvers which allow for custom potential functions similar to those used in Fei Fei Li's work ?

(2) Are there any C++, Python or Matlab implementation that allow for general CRF (not chain CRF - its a special case) ?

Answer 1

I am far from being an expert in Graphical Models but did some reading on them, including CRFs. From what I've read, first order linear chain CRFs are attractive as the inference is efficient and luckily - many problems can be mapped in this paradigm. The problems arises when we would like to work with general graphs.

One of the problems with general graphs (undirected graphical models) is that inference gets intractable. Even if it is possible to exploit structure, the partition function becomes very difficult to compute.

I would like to point to the generalization of undirected graphical models - Energy Based Learning [link] from Yann LeCun. One of the attractive properties is that this partition function is not needed anymore, we are working with energy values. The beauty is that many machine learning solutions can be mapped in energy based learning framework.

This tutorial link on Energy Based Learning is brilliant. Although my background in machine learning is limited, the exposition of the material and the connections to the existing methods is very well laid out.

There is a software called eblearn [link]. I did not manage yet to compile it successfully but I am eager to apply it on my problems. Perhaps it can be useful in your case as well.