Intuitively, the mean is just the average of observations. The variance is how much these observations vary from the mean.

I would like to know why the inverse of the variance is known as the precision. What intuition can we make from this? And why is the precision matrix as useful as the covariance matrix in multivariate (normal) distribution?

Insights please?

  • 1
    $\begingroup$ In computing the likelihood of multi variate Gaussian distribution, precision matrix is more convenient to use. The variance matrix has to be inverted first. $\endgroup$ – user112758 May 8 '16 at 7:35
  • $\begingroup$ To nitpick a bit, the variance is not how far the observation vary from the mean because variance is not expressed in the same units as the mean. "Point $A$ is 8 square meters away from point $B$" is unintelligible... (Tim's answer (+1) should address your specific question I believe.) $\endgroup$ – usεr11852 May 8 '16 at 8:50
  • $\begingroup$ Precision is a measure of, among other things, how likely we are to be surprised by values distant from the mean. $\endgroup$ – Alexis Mar 21 '17 at 2:33
  • $\begingroup$ I think the original question is an excellent one, because I would have thought that precision would be more of a margin of error, e.g., half the width of an uncertainty interval. This would have been more on the square root of variance scale. $\endgroup$ – Frank Harrell Jun 8 '19 at 14:27

Precision is often used in Bayesian software by convention. It gained popularity because gamma distribution can be used as a conjugate prior for precision.

Some say that precision is more "intuitive" than variance because it says how concentrated are the values around the mean rather than how much spread they are. It is said that we are more interested in how precise is some measurement rather than how imprecise it is (but honestly I do not see how it would be more intuitive).

The more spread are the values around the mean (high variance) the less precise is they are (small precision). The smaller the variance, the greater the precision. Precision is just an inverted variance $\tau = 1/\sigma^2$. There is really nothing more than this.

  • 3
    $\begingroup$ There is more than that. Precision is a natural parameter. Variance is not. $\endgroup$ – Neil G May 9 '16 at 20:24

Precision is one of the two natural parameters of the normal distribution. That means that if you want to combine two independent predictive distributions (as in a Generalized Linear Model), you add the precisions. Variance does not have this property.

On the other hand, when you're accumulating observations, you average expectation parameters. The second moment is an expectation parameter.

When taking the convolution of two independent normal distributions, the variances add.

Relatedly, if you have a Wiener process (a stochastic process whose increments are Gaussian) you can argue using infinite divisibility that waiting half the time, means jumping with half the variance.

Finally, when scaling a Gaussian distribution, the standard deviation is scaled.

So, many parameterizations are useful depending on what you're doing. If you're combining predictions in a GLM, precision is the most “intuitive” one.

  • $\begingroup$ Hi Neil, could you provide and example or some links to resources that further explain the "additive" property of the precision when combining two distributions? I am not sure, how to interpret it. $\endgroup$ – Kilian Batzner Jan 7 '18 at 10:52
  • $\begingroup$ @KilianBatzner digitool.library.mcgill.ca/webclient/… page 15. $\endgroup$ – Neil G Jan 7 '18 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.