How can I confidently state the error of a specific test is within a range with a 95% confidence interval I have a test that provides values with known amounts of error. For instance the error on the test
(Value - TrueValue) = -7.62, -9.33, -8.36, -9.79,-10.45, -9.51, -10.83, -10.64, -9.96, -10.30
I want to try to linearly adjust the value to reflect the true value by correcting for the average error. In this case, the average error is -9.679, so adjusting for this the new errors are:
2.059, 0.349, 1.319, -0.111, -0.771, 0.169, -1.151, -0.961, -0.281, -0.621
How can I state with confidence that given any new test, adjusted by the error, its true value is within X given 95% confidence? What sort of test would I want to use? Can anyone provide a resource to read about this topic? Thank you
 A: I'm going to provide a Bayesian approach.
Let $y$ be the error between the measurement and the truth.  If you're willing to make assumptions about the error (maybe that the measurement is biased in some way, that bias needs estimation, and there is additional noise in the process) it may be possible to select a distribution with which to model the errors.  Let's assume for the sake of exposition that a Student t distribution is good enough for our purposes, but any distribution will do.
The model is then
$$ y \sim B + \epsilon $$
Here $B$ is the bias of the measurement and $\epsilon$ is a student t random variable with some degree of freedom, $\nu$, which requires estimation from the data.  There are two sources of variation we need to account for: variation in the estimate of $B$ as well as variation in the estimate of the degrees of freedom for $\epsilon$ (or whatever other parameter is used in your choice or error distribution.  This could be $\sigma$ if you were to use a normal distribution).
To provide an interval estimator, we can leverage the posterior predictive distribution which I have talked about in this post.  IN essence, we will draw values of $B$ and $\nu$ from our posterior and then use those to simulate new data from our likelihood.  Once we do that a couple thousand times, we can summarize the resulting distribution using quantiles to give an interval estimator.
Here is how we might do this with your data
library(brms)

# Data you have provided
errors = c(-7.62, -9.33, -8.36, -9.79,-10.45, -9.51, -10.83, -10.64, -9.96, -10.30)
d = data.frame(errors)

# Fit a Bayesian model with a student t error
model = brm(errors~1, data = d, family = student())

# Sample from the posterior predictive
posterior_predictions = posterior_predict(model, newdata = list(1))

# Summarize a 95% credible interval 
quantile(posterior_predictions, probs = c(0.025, 0.975))

      2.5%      97.5% 
-12.377867  -7.049977 

The result is an interval which contains 95% probability for seeing a new observation given the model specification.  That's our uncertainty in the prediction.  We can also summarize our uncertainty in the bias (much like a confidence interval) in a similar way

B = posterior_samples(model, pars = 'Intercept')[, 1]
quantile(B, probs = c(0.025, 0.975))
     2.5%      97.5% 
-10.439664  -8.974423 

mean(B)
[1] -9.72698


So our estimate of the bias is -9.76 (this will depedn on how many samples you use as well as a few other things not worth mentioning here) and our uncertainty in that bias is given by the credible interval (-10.44 --  -8.97).
The benefit of this approach is integrating over all uncertainty in the model, not just uncertainty in the parameters the model as would be done with a frequentist prediction interval.  However, Bayesian models require a bit of a learning curve to get used to.
A: Put it this way....
You test #1

(Value - TrueValue) = -7.62, -9.33, -8.36, -9.79,-10.45, -9.51, -10.83, -10.64, -9.96, -10.30

And the averaged-error/adjustment is -9.679. If you conduct 1000 tests, you will have 1000 different values of averaged-error/adjustment. Then you can plot the distribution and it would probably be a normal distribution. Then you can calculate the mean and the standard deviation of this distribution, and you can say that the averaged-error/adjustment is mean +/- 2*standard-deviation with 95% confidence.
