Suppose I have a Bayesian network that can be factorized like this:
Each of the variable is a binary and I've got all the tables of conditional probabilistic distribution.
I then simulate from this network and got a synthetic data-set:
A B C D 1 0 0 1 0 1 0 1 1 0 1 1 etc.
I then estimate the parameters (the probabilities) with MLE and Bayesian estimator from the synthetic data and would like to compare these 2 learners by their KL-distances against the true distribution at various sample sizes.
However, I'm not sure how to do it. The book I read about the relative entropy of Bayesian network just has notations in it and it's very hard to follow.