Given a set of data $\{(x_i\pm e_{x,i},\,y_i\pm e_{y,i})\}_i$ (with uncorrelated uncertainties), I want to model it as a multivariate Gaussian function with an unknown mean $\boldsymbol{\mu} $ and a covariance matrix $\boldsymbol{\Sigma} $. What is the correct prior for them? What happens if I assume that mean of both variables are 0?

Context: I've tried MCMC without giving any prior by assuming the likelihood for each pair of points to be a Gaussian with covariance of $$ \boldsymbol{\Sigma}_i = \boldsymbol{\Sigma} + \pmatrix{{e_{x,i}}^2& 0\\0 &{e_{y,i}}^2} $$ but it lead to the correlation coefficient of $\pm$1, so I thought it was the problem of incorrect prior.

  • $\begingroup$ Can you expand on 'correct prior'? Usually the 'correct prior' is the prior distribution that honestly and accurately reflects your (prior) beliefs about the parameters of interest. $\endgroup$
    – jcken
    Jun 17 at 8:18
  • $\begingroup$ @jcken Sorry for unclear wording - I don't know statistics very well. What I want is the least informative prior...or at least something like that. I need to calculate the correlation for any arbitrary set of data in industrial scale, and in most cases I know nothing about them. The intuition I have is that the correlation, given the measurement uncertainties, cannot be exactly 1 or -1. And closer they are to the unity, more unlikely they become. $\endgroup$
    – Hojin Cho
    Jun 17 at 8:43
  • $\begingroup$ Here is a new paper on the topic $\endgroup$ Jun 17 at 19:25

The trick is that there's no need to parametrize the covariance matrix.

Model the usual mean and variance for $X$ using normal and inverse gamma priors. Model the response $Y$ conditionally using a regression model, with normal priors for the intercept and slope, and the inverse gamma for the .

You can derive the covariance term using the following relation:

$$ \beta = \text{Cov}(X,Y)/\text{Var}(X)$$

  • $\begingroup$ Could you provide a proof for the statements you made? I want to follow it so that I could check if this is strictly true in case of multivariate Gaussian fitting. $\endgroup$
    – Hojin Cho
    Jun 21 at 5:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.