I have a Bayesian network DAG structure, and a conditional probability distribution (CPD) for each node. I want to fit the parameters of the CPDs with a Bayesian method, since I have some prior knowledge on the parameters. I implemented my procedure in Stan. But I wonder, would it not be preferable to fit the procedure in JAGS?

My thinking is as follows. The joint distribution in a Bayesian network can be decomposed into the product of conditional probabilities of each node given its parents in the graph.

Let $G$ be a DAG with set of nodes $V$. Let $\theta$ be a vector whose elements are the parameters for the CPDs corresponding to the DAG. Let $P(G|\theta)$ be the joint probability of all the nodes in the graph.

\begin{align} P(G|\theta) = \prod_{X \in V} P(X| \text{Pa}(X), \theta_X) \end{align} where $X \in V$ is a node, $Pa(X) \subset V$ is $X$’s parents in $G$, $\theta_X$ are the parameters of X's CPD, and $P(X| \text{Pa}(X), \theta_X)$ is the CPD for a $X$.

In both JAGs and Stan, I could specify a prior on $\theta_x$ (in the parameter block for Stan) and the following sampling statement for each node X \begin{align} X \sim P(X| \text{Pa}(X), \theta_X) \end{align} The ordering of the sampling statements would correspond to the ordering of the DAG.

Stan uses Hamiltonian MCMC, which tries to optimize a potential energy function based on the joint posterior of all the elements in $\theta$. But in a Bayesian network the parameters are independent, and therefore the posterior of $\theta$ would just split up into $\pi(\theta_x \vert X, Pa(X))$, a bunch of marginal posteriors. I don't know much about how JAGS works, other than it is a Gibbs sampler. It seems to me a Gibbs sampler would work better in this case, especially for DAGs with many nodes.

Anyone have any thoughts on the matter?

  • $\begingroup$ In theory, Gibbs should work well if you're able to write the conditional updates for each $\theta$ that you have. In practice, the No-U-Turn is much faster at exploring a posterior even when you have an analytic form for the conditional updates. It really shows in higher dimensions. Take a look at the experiments in the original NUTS paper by Hoffman and Gelman: jmlr.org/papers/volume15/hoffman14a/hoffman14a.pdf. $\endgroup$ – syclik Jun 21 '17 at 16:38
  • $\begingroup$ Regarding HMC: just wanted to mention that it's not optimizing a potential energy function. $\endgroup$ – syclik Jun 21 '17 at 16:38
  • $\begingroup$ Actually, JAGS performs Gibbs sampling only in selected cases, hence can't be really used to assess Gibbs sampling, in my opinion. $\endgroup$ – altroware Sep 3 '17 at 12:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.