Probabilistic Graphical Model - Modelling a Bayesian Network on Real Life Data

Recently, I have started studying about Probabilistic Graphical Models (PGMs). While the examples provided in the textbook essentially convey the message of what and how things are happening, I am finding it particularly difficult to solve almost ANY real-life problem using PGMs, especially Bayesian Networks, and obviously my proximity of problems, i.e. image processing.

Of course, I am aware of the use-cases, there are published papers which are using PGMs and Image Processing together, but please bear with me for the the example provided below.

While working on Handwritten Character Recognition, I came across this paper. It uses Convex Hull based features. While not following the paper completely, I calculated 20 odd features of the characters.

Now comes the meat of the problem. These values are continuous, in a sense they can take any value from 0 to size of the image.

I can create bins of the length one, or more. Let us proceed with the assumption that the bin is of length 1, so that a 64X64 image can have each feature in range of 0 to 63. So essentially, I would have 20 Random Variables, each having a value between 0 to 63.

Now comes the part of creating a model. From the domain knowledge, I am quite certain that each of the feature have equal significance in inferring the character. Hence, a model - 20 Random Variables (Features) are parents of 1 Random Variable (Character). Though this works in theory, this is clearly not practical, as there would be 63^20 possible combinations trying to infer a single value. This simply blows off a memory.

As a matter of fact, I tried creating this, using pgmpy library, and as expected it gave a memory error.

Now even if I increase the bin size, the number of discrete values of each Feature Random Variable cannot be less than 10 for all practical purposes, in that they won't be features anymore if I reduce the categories to less than 10. Pl note I have 26 classes (English Characters). Still, I would have 10^20 numbers being crunched to infer a probability of character.

In the aforementioned paper, the 20 features are actually versions of 5 features, changing the reference point of the image from Left, Right, Top and Bottom. Hence, I can form a network as below:

But in that case, a questions are

1. How many discrete values to include for each of the Top, Left, Bottom and Right Node?

2. How to compute these values, as these are completely imaginary nodes. For f1 to f20, I am going to have a data to train my model. Are there functions designed only for such purpose?

Also, note that - I applied PCA, to be specific sk-learn's RandomizePCA, but it gives 1% accuracy.

I have tested that these features indeed work - using Decision Tree Classifier, it produces 75% accuracy.

But Still, being in doubt, I present the statistic of my data.

Array Shape: (260, 20)

Vertical Means: [  58.46923077   99.41923077   15.5          32.65769231   98.19230769
58.62307692  106.73846154   26.27307692   25.36538462  101.07307692
69.05769231   92.02307692   26.37307692   28.78076923   96.69230769
60.92307692   91.51538462   22.13076923   29.42307692   98.00384615]

MinAlongVerAxis: [ 1  4  1  0 39  1  4  1  0 68  1  5  1  0 19  1  5  1  0 29]

MaxAlongVerAxis: [158 170  98 135 170 151 163 103 127 150 174 156 111  90 174 159 161  68
114 176]

StdDevAlongVerAxis: [ 46.13178255  30.49938863  16.55701106  24.57160139  13.32931769
42.77917635  33.77152962  25.57339449  26.05263079  11.8758986
43.45493398  35.90510257  25.68573471  27.30669852  17.89535658
42.46040968  38.09607987  18.5824104   29.54054981  17.35478167]

ModeAlongVerAxis: [[  8  94   1   1  95   5 124   1   1 100   4 106   1   1  96   3  48   1
1 100]]
ModeOccurences: [[12  7 33 13 17  8  8 33 49 17 10  7 31 52 20 12  5 44 56 14]]


TL;DR

1. How to discretize continuous values for PGM, so that it is computationally managable?
2. If few imaginary nodes are computed, how to come up with their values? Obviously some function has to be there, but which and why?
3. Are there methods to make above mentioned particular example work using Bayesian Network?
• I'm more interested in how you're using a wooden board as a whiteboard. Commented May 13, 2016 at 19:27
• In the paper you linked, they use an MLP classifier, with a hidden layer, purportedly with a 99.45% accuracy. Would this not suffice for your purposes? I don't think a bayesian network is the way to go here, as problems such as this are easily solved using either neural nets or basic classifiers as mentioned in the paper. Commented May 13, 2016 at 19:29
• Point is not solving the problem. Point is to learn to use Bayesian Network and in turn PGM. If not Bayesian, what other PGM is suitable here, and of course, why? Commented May 14, 2016 at 5:42
• Just as a correction though, 99.45% accuracy is for Bangla Numerals, for Bangla Characters they have achieved 76.86% accuracy. Irrespective of that, I am working on English Handwritten Characters, my dataset is different, so potentially, both are different problems, and hence not directly comparable Commented May 14, 2016 at 5:50
• I don't think you're going to have a fun time implementing the above as a PGM because the dimensionality seems quite high. Instead, try using a markov network, like the one found here: papers.nips.cc/paper/2397-max-margin-markov-networks.pdf Commented May 14, 2016 at 15:53