9
$\begingroup$

How do you calculate the appropriate prior if you have the measurement error of an instrument? This paragraph is from Cressie's book "Statistics for Spatio-Temporal Data":

It is often the case that some prior information is available regarding the measurement-error variance, allowing a fairly informative parameter model to be specified. For example, if we are assuming conditionally independent measurement errors that are iid $Gau(0, \sigma_{\epsilon}^2)$, then we should specify an informative prior for $\sigma_{\epsilon}^2$. Say we were interested in ambient air temperature, and we saw that the instrument manufacturer’s specifications indicated an “error” of $±0.1°C$. Assuming that this “error” corresponds to 2 standard deviations (an assumption that should be checked!), we might then specify $\sigma_{\epsilon}^{2}$ to have a prior mean of $(0.1/2)^2 = 0.0025$. Because of the instrument manufacturer’s specification, we would assume a distribution that had a clearly defined and fairly narrow peak at 0.0025 (e.g., inverse gamma). In fact, we could just fix at 0.0025; however, the data-model error may have other components of uncertainty too (Section 7.1). To avoid possible identifiability problems with process-model error, it is very important that modelers reduce the uncertainty in as much as the Science allows, including doing side studies designed to have replicated data.

Does anyone know what is the general procedure to obtain the values of a prior as described above (although the paragraph only refers to obtaining the prior mean)?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
6
$\begingroup$

Two standard methods are

  1. Consult the "instrument maker's specifications," as indicated in the quotation. This is usually a crude fall-back to be used when no other information is available, because (a) what the instrument maker really means by "accuracy" and "precision" is often indeterminate and (b) how the instrument responded when new in a test lab was likely much better than it performs when used in the field.

  2. Collect replicate samples. In environmental sampling there are about a half dozen levels at which samples are routinely replicated (and many more at which they could be replicated), with each level used to control for an assignable source of variation. Such sources may include:

    • Identity of the person taking the sample.
    • Preliminary procedures, such as bailing wells, taken before obtaining a sample.
    • Variability in the physical sampling process.
    • Heterogeneity within the sample volume itself.
    • Changes that might occur when preserving and shipping a sample to a laboratory.
    • Variations in preliminary laboratory procedures, such as homogenizing a physical sample or digesting it for analysis.
    • The identify of the laboratory analyst(s).
    • Differences between laboratories.
    • Differences between physically distinct instruments, such as two gas chromatographs.
    • Drift in instrument calibration over time.
    • Diurnal variation. (This may be natural and systematic but can appear random when sampling times are arbitrary.)

A full quantitative assessment of components of variability can only be obtained by systematically varying each of these factors according to a suitable experimental design.

Usually only the sources believed to contribute the most variability are studied. For instance, many studies will systematically split a certain portion of the samples once they have been obtained and ship them to two different laboratories. A study of the differences among the results of those splits can quantify their contribution to the measurement variability. If enough such splits are obtained, the full distribution of measurement variability can be estimated as a prior in a hierarchical Bayesian spatio-temporal model. Because many models assuming Gaussian distributions (for each of calculation), obtaining a Gaussian prior eventually comes down to estimating the mean and variance of the differences between the splits. In more complicated studies, which aim to identify more than one component of variance, this is performed with the Analysis of Variance (ANOVA) apparatus.

One of the benefits of even thinking about these issues is that they help you identify ways to reduce or even eliminate some of these components of error (without ever having to quantify them), thereby getting closer to Cressie & Wikle's ideal of "reducing the uncertainty as much as the science allows."

For an extended worked example (in soil sampling), see

Van Ee, Blume, and Starks, A Rationale for the Assessment of Errors in the Sampling of Soils. US EPA, May 1990: EPA/600/4-90/013.

Improve this answer
$\endgroup$
8
  • 2
    $\begingroup$ The issue here, Robert, is that sometimes somebody will report a standard deviation of for the estimate; other times they will report twice it (whence the division by two) or a two-sided confidence interval; and sometimes even something else; so there is no definite rule for converting statements of accuracy and precision into priors: you must consult the footnotes and other technical details to figure out exactly what the numbers represent. The standard error of an estimate, being a function of the size of the sample used, is irrelevant for this purpose BTW. $\endgroup$
    – whuber ♦
    Commented Oct 2, 2014 at 2:28
  • 1
    $\begingroup$ Got it. Let me change the focus to your second case. If I repeat an experiment a couple of times and obtain measurements $m_{1}$ and $m_{2}$, how can I use this information to inform the mean and variance for a prior distribution? You suggested something like $m_{1}- m_{2}$ for several splits, right? Therefore, I'd have a mean of a measurement error $m_{\epsilon}$ and a sample standard deviation $\sigma_{\epsilon}$. Is that enough to include it in a prior $N(m_{\epsilon}, \sigma_{\epsilon}^{2})$? $\endgroup$
    – r_31415
    Commented Oct 2, 2014 at 3:22
  • 1
    $\begingroup$ You cannot assess accuracy with splits: for that, you need to measure samples of known values. (Laboratory spikes and spiked duplicates are used for this.) That will determine the mean. Usually this is handled when calibrating the measurement process, so the mean is taken to be zero. The variance is estimated with the usual ANOVA formulas. You could use that to specify a prior on the corresponding component of the measurement system. $\endgroup$
    – whuber ♦
    Commented Oct 2, 2014 at 3:40
  • 2
    $\begingroup$ Not so: The reference I gave is US EPA guidance that has been around for a quarter century and plenty of more recent guidance builds on its ideas. I once used this approach in a federal court case to evaluate the effect of measurement error on contour lines drawn (based on a geostatistical predictor) to delineate a contaminant plume: the measurement error was larger than the concentration used to bound the plume! (In other words, the uncertainty in the plume delineation was essentially infinite.) $\endgroup$
    – whuber ♦
    Commented Oct 2, 2014 at 3:50
  • 1
    $\begingroup$ Really nice. By the way, I meant to say that priors are usually set without taking much care. I have seen this more prominently in Bayesian modeling and machine learning maybe because a guess is often enough to produce decent results. $\endgroup$
    – r_31415
    Commented Oct 2, 2014 at 3:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.