0
$\begingroup$

In structure learning, there are score-based methods which rely on information criteria such as BIC or AIC.

BIC, specifically, is defined as:

$$ BIC = k \ln(n) - 2\ln\left(\hat{L}\right) $$

Where $k$ is the number of parameters in the model, $n$ is the number of training examples and $\hat{L}$ is the likelihood function associating the model itself with observed data $x.$

In the case of structure learning, we are trying to determine how well a bayesian network approximates the underlying causal structure in the observed data. The bnlearn R package implements such calculations in its methods and, as far as I can tell, the log-likelihood is usually the preferred likelihood function, as it is supposed to be easier to compute.

So my main question here is: how is $\hat{L}$ calculated in the context of bayesian networks? I understand the "concept" (I think), but I can't seem to visualize a practical way of implementing that. What would be the most straight-forward way of representing such calculation?

$\endgroup$
0
$\begingroup$

Conceptually, one could use the "standard" way of calculating likelihood, see How to calculate the likelihood function in conjunction with the rule of product decomposition i.e., chain rule for the network. So the joint-probability for the entire network reads $$P(x_{1}, x_{2}, ..., x_{n}) = {\Large \Pi} P(x_{i}|x_{parent})$$

$x_{parent}$ is the parent of $x_{i}$. One would need to incorporate this in formulating "standard" likelihood $L(\theta|x_{1}, x_{2}, ..., x_{n})$, $\theta$ being the parameters.

$\endgroup$
2
  • $\begingroup$ I understand the concept. But how do you calculate a given $P(x_{1}, x_{2}, ..., x_{n}) = {\Large \Pi} P(x_{i}|x_{parent})$? $\endgroup$ – eduardokapp Jun 14 at 12:07
  • $\begingroup$ Construct contingency tables (Conditional probability table) for each node. See wikipedia sprinkler example. However, please note that to compute $L$ we would need to do maximum likelihood estimation and it is involved. Daphne Koller's lecture on MLE on graphical models might give better idea, here. $\endgroup$ – Mehmet Suzen Jun 14 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.