I think this might be a basic scenario but I still struggle to find an appropriate methodology for my problem. Consider the following scenario:

There is a small moving object on an $xy$-millimeter-grid paper. I have a measurement system $A$ trying to estimate the $xy$ position for example by using IMU measurements. My "ground truth" measurement system $B$ could be a downwards facing laser mounted on the object in combination with me reading the laser point positions from the grid paper.

Now I collect measurements at $100$ timestamps with both system $A$ and system $B$. To estimate the uncertainty for system $A$ calculate the error between the $xy$ position estimated by $A$ and the "ground truth" reading from $B$. Over the 100 samples I can calculate the mean error $\mu$ and the std $\sigma$, for example $\mu=8.4\,mm$ and $\sigma=5\,mm$. Then I can use for example a $t$-distribution to calculate a $k=2$ confidence interval with which I can characterize the estimated true mean of system $A$'s measurement error, e.g. $\mu_{est}=8.4\,\pm\,3.5 \, mm$.

What if I want to take into account inherent uncertainties from my "ground truth" system $B$ into that confidence interval? For example, what if I know that the laser has a specific uncertainty or what if I know that my readings from the grid paper have a certain deviation as the laser point is at least $1\,mm$ thick? How would I include such knowledge into the confidence interval of measurement system $A$ in order to give a more precise uncertainty estimation?


Below I will use $X$ and $Y$ to denote random variables related to systems $A$ and $B$. You could frame the problem in terms of a matched pairs test, a two-sample test, or a prediction. That is, you could think in terms of


and perform inference on $\Delta$, the mean difference, where $\hat{\sigma}_{X-Y}$ is the estimated standard deviation of $X-Y$. You could also think in terms of


and perform inference on $\theta$, the difference in means, where $\hat{\text{se}}_{\bar{X}-\bar{Y}}$ is the standard error of $\bar{X}-\bar{Y}$. Lastly, you could assume the two random variables have the same mean and think in terms of




These quantities allow you to perform predictive inference (predictive p-values, prediction intervals) for hypotheses around a future $\bar{Y}=\bar{y}$ based on an observed $\bar{X}=\bar{x}$ or predictive inference on a future $Y_i=y_i$ based on an observed $X_i=x_i$. These aren't the only pivotal quantities for inference and prediction, but they should look familiar and account for the uncertainty in having measured both $X$ and $Y$.

I hope this helps! See my papers on confidence distributions and prediction intervals.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.