Intuitively, the mean is just the average of observations. The variance is how much these observations vary from the mean.
I would like to know why the inverse of the variance is known as the precision. What intuition can we make from this? And why is the precision matrix as useful as the covariance matrix in multivariate (normal) distribution?
Insights please?
Precision is often used in Bayesian software by convention. It gained popularity because gamma distribution can be used as a conjugate prior for precision.
Some say that precision is more "intuitive" than variance because it says how concentrated are the values around the mean rather than how spread they are. It is said that we are more interested in how precise is some measurement rather than how imprecise it is (but honestly I do not see how it would be more intuitive).
The more spread are the values around the mean (high variance), the less precise they are (small precision). The smaller the variance, the greater the precision. Precision is just an inverted variance $\tau = 1/\sigma^2$. There is really nothing more than this.
Precision is one of the two natural parameters of the normal distribution. That means that if you want to combine two independent predictive distributions (as in a Generalized Linear Model), you add the precisions. Variance does not have this property.
On the other hand, when you're accumulating observations, you average expectation parameters. The second moment is an expectation parameter.
When taking the convolution of two independent normal distributions, the variances add.
Relatedly, if you have a Wiener process (a stochastic process whose increments are Gaussian) you can argue using infinite divisibility that waiting half the time, means jumping with half the variance.
Finally, when scaling a Gaussian distribution, the standard deviation is scaled.
So, many parameterizations are useful depending on what you're doing. If you're combining predictions in a GLM, precision is the most “intuitive” one.
Here is my attempt at an explanation:
A) An intuition for precision can be found in the context of measurement error. Suppose you are measuring some quantity of interest with some measurement instrument (e.g., measuring a distance with measuring tape). If you were to take several measurements of the quantity of interest with the same measurement instrument, you will likely end up with variation in the results i.e. measurement error. These errors are often well approximated by a normal distribution. The precision parameter of a normal distribution tells you how "precise" your measurements are in the sense of having larger or smaller errors. The larger the precision, the more precise your measurement, and thus the smaller your errors (and vice-versa).
B) The reason that precision matrices are sometimes preferred over covariance matrices is due to analytical and computational convenience: they are simpler to work with. This is why normal distributions were classically parameterized via the precision parameter in the Bayesian context before the computer revolution when calculations were done by hand. The parameterization remains relevant today when working with very small variances as it helps to address underflow in numerical computations.
The simplicity of the alternative can also be illustrated by comparing the densities of both parameterizations. Notice below how the use of $\tau = \frac{1}{\sigma^2}$ eliminates the need to divide by a parameter. In a Bayesian context (when parameters are treated as random variables) division by a parameter can make calculating posterior distributions painful.
$$p_Y(y; \mu, \sigma) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{1}{2}(\frac{y-\mu}{\sigma})^2}$$
$$p_Y(y; \mu, \tau) = \sqrt{\frac{\tau}{2\pi}}e^{-\frac{1}{2}\tau(y - \mu)^2}$$
