# Bayesian prior choice for multivariate Gaussian distribution

Consider spatially distributed data $\boldsymbol x = \{x(t_1),x(t_2),\dots,x(t_n)\}$ which is described by a multivariate Gaussian distribution with mean $\mu$, standard deviation $\sigma$ and stationary correlation function which depends on the absolute distance $\tau = |t_1-t_2|$ between points:

$$\rho(\tau) = \exp\left(-\frac{2\tau}{\theta}\right)$$

Which produces a correlation matrix $\boldsymbol \rho(\theta)$.

$$\boldsymbol x \sim \mathcal{N}(\mu,\sigma^2 \boldsymbol \rho)$$

Given samples of $\boldsymbol x$ the aim is to predict $\mu$, $\sigma$ and $\theta$. Under the Bayesian approach I believe that a joint prior distribution must be specified. Since I want to use Markov Chain Monte Carlo to sample from the posterior I believe the prior should be proper. My approach is to specify marginal distributions $P(\mu)$, $P(\sigma)$, $P(\theta)$ and take the joint prior to be given by:

$$P(\mu,\sigma,\theta) = P(\mu)P(\sigma)P(\theta)$$

Is it reasonable to take the priors to be independent?

Can I take the priors to be uniform if am confident the parameter values fall in certain intervals i.e.

$$\mu \sim U[-100,100]$$ $$\sigma \sim U[0,1000]$$ $$\theta \sim U[0,1000]$$

Or should I use a Cauchy for $\mu$ and the folded-Cauchy for $\sigma$, $\theta$?

• What's this for, kriging? – Jon Feb 2 '17 at 18:46
• Often a inverse Gamma prior is assumed for $\sigma$. Do you have any specific reasons (read: prior knowledge/belief) to prefer a uniform prior? – DeltaIV Feb 2 '17 at 20:41
• @DeltaIV Jarad's comment has made me realise that a uniform prior is not non-informative. Do I believe that $Pr(0 \leq \sigma \leq 100) = Pr(900 \leq \sigma \leq 1000)?$. No. However I don't like inverse Gamma due to its behaviour around zero (see here). I think I will go with the folded Cauchy. – Fundamental Engineer Feb 2 '17 at 20:45
• Any prior can be used, there is no way one can recommend one prior as been better than another on a general basis. – Xi'an Feb 2 '17 at 21:39

I believe you are using uniform priors in an attempt to be non-informative, but I believe your priors are more informative than you would like them to be. Do you really believe $\sigma$ or ($\theta$) is equally likely to be between (0,1) as (999,1000)? And that there is absolutely no probability whatsoever that these parameters are above 1000?
My suggestion is $\sigma \sim Ca^+(0,c_\sigma)$ (i.e. a folded-Cauchy or half-Cauchy), $\theta \sim Ca^+(0,c_\theta)$, an improper uniform for $\mu$ and all independent. You will need to choose $c_\sigma$ and $c_\theta$ for your problem, but since you are already "confident" the parameter values fall into the regions given, you can just choose these values so the probability outside those regions is small. I prefer heavy-tailed distributions like the half-Cauchy because the data can more easily overwhelm the prior in case you were wrong in your prior assumptions. If you really want the prior to be proper, you could use a Cauchy for $\mu$ and specify a location and scale parameter.