How to determine the precision of a measurement using a "perfect" reference method?

Question

I developed an experimental method that measures certain material property. I want to see how precise my method is, so I have a confidence interval, or other metric. To do this, I'm comparing it with an established (and theoretically "perfect") reference method.

Sadly, one method is destructive, and due to external issues, I couldn't do multiple measurements with the other method.

From linear regression I can see that there's linear relation with $R^2=0.9964$, so my method isn't crap.

But when I measure a new sample, I want to be able to say that measurement is e.g. $0.22 \pm0.005$

I have a very small dataset that looks as below. It is impossible to make more measurements. What calculation should I make? I'm not a real statistician, so I need a bit of demonstration/step-by-step approach to avoid misunderstanding.

Sample ID	Reference method	Method Under Investigation
1	0.235	0.219
2	0.06	0.04
3	0.055	0.034
4	0.149	0.145
5	0.272	0.26

Answer 1

You seem to want a prediction interval.

A confidence interval gives a range of plausible values of the average $y$ for a given $x$, while a prediction interval gives a range of plausible values of the observed $y$ for a given $x$.

In R software, you can do the following.

set.seed(2023)
x <- c(0.219,
       0.04,
       0.034,
       0.145,
       0.26
)
y <- c(0.235,
       0.06,
       0.055,   
       0.149,   
       0.272)
L <- lm(y ~ x) # regress the perfect measurement on the imperfect measurement
pred.int =  predict(object = L, 
                    newdata = data.frame(x = x), 
                    interval = "predict",
                    level = 0.95)
pred.int

> pred.int
fit        lwr        upr
1 0.23080914 0.20559627 0.25602201
2 0.05810088 0.03209494 0.08410681
3 0.05231177 0.02604125 0.07858229
4 0.15941019 0.13563641 0.18318397
5 0.27036802 0.24339181 0.29734423

The chart tells you that the estimated/predicted value is in the first column, while the lower and upper limits of the prediction interval are in the second and third columns, respectively. The "level" argument is analogous to the confidence level, so these would be $95\%$ prediction intervals (since level = 0.95).

In math, the prediction intervals are calculated by the following:

$$ \hat y_i \pm t_{\alpha/2, n-p}s\sqrt{1 + x_i\left(X^TX\right)^{-1}x_i} $$

$\alpha$ is one minus your desired level, so $\alpha = 0.05$ for a $95\%$ interval, $\alpha = 0.1$ for a $90\%$ interval, etc.

$n$ is the sample size, so $5$ in this case.

$p$ is the number of parameters in the regression equation, so $2$ in this case (slope and intercept).

$t_{\alpha/2, n-p}$ is the critical value for a two-sided confidence interval at the $(1-\alpha)100\%$-level, for a $t$-distribution with $n-p$ degrees of freedom.

$s$ is the estimated error variance from the regression.

$x_i$ is a vector of $1$ followed by the imperfect measurement.

$y_i$ is the predicted value.

$X$ is the model matrix, $X = \left( \begin{matrix} 1 & 0.219\\ 1 & 0.04 \\ 1 & 0.034 \\ 1 & 0.145 \\ 1 & 0.26 \end{matrix} \right)$, in this case.

The $1$ is added inside the square root so that the prediction interval never shrinks to have a length of zero as the sample size increases, which contrasts with a confidence interval that does shrink to have zero length as the sample size goes to infinity.