# MCMC for structured matrix

I'm working with a Gaussian random field, which can be described by joint pdf as follows:

$$X\mid \sim N\left ( \mu, \sigma^{2} \left ( I-C \right )^{-1}\right )$$

where $C$ is a structured matrix, $c_{ij}=0$ if $i$ and $j$ is not neightbours. I'm working with the Bayesian inference on such fields. So the main question is about how to sample such structured matrix? Every draw has to be of the same structure as previous, i.e. zeros for not neighbours and the $I-C$ has to be positive definite. Although there are papers on how to generate positive definite matrix by MCMC, but I have not found any about how to draw a structured matrix.

• Can you give more detail about the state space of the MCMC chain?
– Cyan
Aug 23, 2012 at 14:26
• I do not understand the question: if $C$ is structured the way you indicate, the only unknown part is $\sigma^2$. In a Bayesian framework, this is all you need to sample. (Besides $\mu$.) Aug 26, 2012 at 11:08
• Probably I have provided not enough information on this. The matrix C also contains unknown parameters and in some places (for non-neighbourg vertices) this matrix contains zeros. However, I have found the solution. Aug 26, 2012 at 12:45
• @Tomas Care to share your solution with us? Aug 26, 2012 at 18:03
• @ David J. Harris See my answer below. Aug 26, 2012 at 19:24

First of all, lets simplify things. assume, that we have a zero-mean Gaussian Markov field with an unknown covariance matrix $\Sigma$ $$X\mid \Sigma\sim N\left ( 0,\Sigma \right )$$ Conditional independence of two variables $X_{i}$ and $X_{j}$, given the rest, is garantued iff the $(i,j)$ element of the precision matrix (sometimes refered as concentraition matrix) $Q=\Sigma^{-1}$ is equal to zero.
This precision matrix has to be invertable and positive definite. So, in order to have a posterior samples of $\Sigma$ or $Q$ we have to have a way to randomly draw such structured matrices.
Håvard Rue have advised to look for information about so called G-Wishart distribution and a. Lenkoski work on it. The expression for this distribution is as follows (up to a normalizing constant): $$f\left ( K|D \right )\propto \left ( det(K)) \right )^\frac{\delta-2}{2}exp\left \{ -tr\left ( KD \right ) \right \}$$ K is a matrix with the structure we need and is positive definite. Now lets put G-Wishart distribution as prior on $Q$, so that posterior will be also G-Wishart distribution: $$Q|X\sim W_{G}\left ( \delta +n, D+\sum_{k=1}^{n}X_{k}^{T}X_{k} \right )$$ To obtain random samples I have employed the iterated proportional scaling algorithm and Gibbs sampler as described in the paper of Lentoski and Dobra (see reference below). So that each random draw is a positive definetely matrix with zero entries where variables are conditionaly independent.