23
$\begingroup$

Intuitively, the mean is just the average of observations. The variance is how much these observations vary from the mean.

I would like to know why the inverse of the variance is known as the precision. What intuition can we make from this? And why is the precision matrix as useful as the covariance matrix in multivariate (normal) distribution?

Insights please?

$\endgroup$
  • 1
    $\begingroup$ In computing the likelihood of multi variate Gaussian distribution, precision matrix is more convenient to use. The variance matrix has to be inverted first. $\endgroup$ – user112758 May 8 '16 at 7:35
  • $\begingroup$ To nitpick a bit, the variance is not how far the observation vary from the mean because variance is not expressed in the same units as the mean. "Point $A$ is 8 square meters away from point $B$" is unintelligible... (Tim's answer (+1) should address your specific question I believe.) $\endgroup$ – usεr11852 May 8 '16 at 8:50
  • $\begingroup$ Precision is a measure of, among other things, how likely we are to be surprised by values distant from the mean. $\endgroup$ – Alexis Mar 21 '17 at 2:33
  • 1
    $\begingroup$ I think the original question is an excellent one, because I would have thought that precision would be more of a margin of error, e.g., half the width of an uncertainty interval. This would have been more on the square root of variance scale. $\endgroup$ – Frank Harrell Jun 8 '19 at 14:27
26
$\begingroup$

Precision is often used in Bayesian software by convention. It gained popularity because gamma distribution can be used as a conjugate prior for precision.

Some say that precision is more "intuitive" than variance because it says how concentrated are the values around the mean rather than how spread they are. It is said that we are more interested in how precise is some measurement rather than how imprecise it is (but honestly I do not see how it would be more intuitive).

The more spread are the values around the mean (high variance), the less precise they are (small precision). The smaller the variance, the greater the precision. Precision is just an inverted variance $\tau = 1/\sigma^2$. There is really nothing more than this.

| cite | improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ There is more than that. Precision is a natural parameter. Variance is not. $\endgroup$ – Neil G May 9 '16 at 20:24
  • $\begingroup$ What is a natural parameter? Edit: it's in your answer. :) $\endgroup$ – mdeff Jul 31 at 16:50
9
$\begingroup$

Precision is one of the two natural parameters of the normal distribution. That means that if you want to combine two independent predictive distributions (as in a Generalized Linear Model), you add the precisions. Variance does not have this property.

On the other hand, when you're accumulating observations, you average expectation parameters. The second moment is an expectation parameter.

When taking the convolution of two independent normal distributions, the variances add.

Relatedly, if you have a Wiener process (a stochastic process whose increments are Gaussian) you can argue using infinite divisibility that waiting half the time, means jumping with half the variance.

Finally, when scaling a Gaussian distribution, the standard deviation is scaled.

So, many parameterizations are useful depending on what you're doing. If you're combining predictions in a GLM, precision is the most “intuitive” one.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Hi Neil, could you provide and example or some links to resources that further explain the "additive" property of the precision when combining two distributions? I am not sure, how to interpret it. $\endgroup$ – Kilian Batzner Jan 7 '18 at 10:52
  • $\begingroup$ @KilianBatzner digitool.library.mcgill.ca/webclient/… page 15. $\endgroup$ – Neil G Jan 7 '18 at 19:59
  • $\begingroup$ "That means that if you want to combine two independent predictive distributions (as in a Generalized Linear Model), you add the precisions. Variance does not have this property." - Why wouldn't variance have this property if it's simply the inverse of precision? $\endgroup$ – skeller88 Jul 16 at 21:46
  • $\begingroup$ @skeller88 because additivity doesn't imply that reciprocals are additive. $\endgroup$ – Neil G Jul 16 at 21:51
  • 1
    $\begingroup$ @mdeff Girdhar, Neil. The informative message model. Diss. McGill University, 2013. $\endgroup$ – Neil G Aug 1 at 13:31
0
$\begingroup$

Here is my attempt at an explanation:

A) An intuition for precision can be found in the context of measurement error. Suppose you are measuring some quantity of interest with some measurement instrument (e.g., measuring a distance with measuring tape). If you were to take several measurements of the quantity of interest with the same measurement instrument, you will likely end up with variation in the results i.e. measurement error. These errors are often well approximated by a normal distribution. The precision parameter of a normal distribution tells you how "precise" your measurements are in the sense of having larger or smaller errors. The larger the precision, the more precise your measurement, and thus the smaller your errors (and vice-versa).

B) The reason that precision matrices are sometimes preferred over covariance matrices is due to analytical and computational convenience: they are simpler to work with. This is why normal distributions were classically parameterized via the precision parameter in the Bayesian context before the computer revolution when calculations were done by hand. The parameterization remains relevant today when working with very small variances as it helps to address underflow in numerical computations.

The simplicity of the alternative can also be illustrated by comparing the densities of both parameterizations. Notice below how the use of $\tau = \frac{1}{\sigma^2}$ eliminates the need to divide by a parameter. In a Bayesian context (when parameters are treated as random variables) division by a parameter can make calculating posterior distributions painful.

$$p_Y(y; \mu, \sigma) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{1}{2}(\frac{y-\mu}{\sigma})^2}$$

$$p_Y(y; \mu, \tau) = \sqrt{\frac{\tau}{2\pi}}e^{-\frac{1}{2}\tau(y - \mu)^2}$$

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.