# Error bars, linear regression and “standard deviation” for point

I have a set of experimental data points. I performed the measurements in triplicate, for each of the point of the data set. Therefore, I can draw each data point with the standard deviation of each triplicate. See the picture attached.

In experimental sciences, it is common to report a value with its standard deviation. Ex: a mean, +/- the std.

I can calculate a linear regression for the data set. If I have the equation of the linear regression, I can calculate x for any y. Let's take y=50 -> x=11.69

Now, is there a way to evaluate the "dispersion" of this extrapolated point ? Something like 11.69 +/- something.

I know it should be the other way around, like 50 +/- something for x=11.69, but then I could use the equation to transform it to x.

Basically what I'm asking: is there a global "standard deviation" for a complete linear regression ?

EDIT: When I say "any y", I mean that y will not be an experimental value. I choose it to be 50.

Your dream of a "global SD" for estimated errors in $x$ given a value of $y$ is not possible.

If what you care about is the SD of a prediction of $x$ given a value of $y$, then what you should examine is the square root of equation (10) of the linked reference. The same result is provided in equation 5.25 of an online analytical chemistry textbook that I find more generally useful and of whose content you should, as a chemist, be aware. Say that you generate a standard curve with known values of $x$ and measured values of $y$. The slope of the standard curve by linear regression was $\beta_1$, and the standard deviation about the regression for the standard curve was: $$s_r=\sqrt{\frac{\sum_i(y_i-\hat{y_i})^2}{n-2}}$$ where $y_i$ are the individual observed values in the standard curve, $\hat{y_i}$ are the corresponding individual predicted values from the regression and $n$ is the number of observations making up the standard curve.

You then make $m$ subsequent measurements of $y$ on a sample with unknown $x$ to estimate that unknown value of $x$, obtaining a mean value $\overline{Y}$. The standard deviation of the estimated value in $x$ based on this value of $\overline{Y}$ is then:

$$s_x=\frac{s_r}{\beta_1}\sqrt{\frac{1}{m}+\frac{1}{n}+\frac{(\overline{Y}-\bar{y})^2}{\beta_1^2\sum_i(x_i-\bar{x})^2}}$$

In this equation individual $x$ values for generating the standard curve were $x_i$ with mean value $\bar{x}$; the corresponding $y$ values for the standard curve had mean value $\bar{y}$.

Note that this standard deviation increases as the observed mean value $\overline{Y}$ for the unknown sample moves away from the mean value $\bar{y}$ determined when generating the standard curve. There thus is no global SD value for all $x$ predicted on the basis of measuring $y$. It must be calculated anew for each unknown sample. You will obtain the most precise results if your unknowns are close to the mean value of the standards used to generate the standard curve.

Happily, there is an R package chemCal that can perform all these calculations in even a more general setting where different observations have different weights. The function of interest in that package is inverse.predict.

• 1) I'm not sure I'm in the case you mentioned. I won't perform $m$ measurements of an unknown sample. I established the model above, and I want to extrapolate the value of x for y=50 (ordinate are percentages), and have a std-like value for the extrapolated x. So maybe I can neglect the last term of the root ? 2) Are you sure you wrote eq 10 ? The last term of the root is different from the pdf I provided – Rififi Apr 11 '16 at 7:50
• My bad, eq 10 and eq 5.25 are different but should give the same result. I didn't notice $\beta_{1}^2$ in the fraction of the square root. – Rififi Apr 11 '16 at 9:15
• If you are using percentages then you probably should not be using a standard linear regression. Depending on the nature of the original data, you should be doing some type of logistic or beta regression, otherwise the residual errors will probably not meet the assumptions upon which all of these standard error estimates are based. With a valid linear regression for the standard curve and a value of 50 for $y$ precisely known, that is equivalent to an infinite number $m$ of measurements, so the first term under the root is 0. With percentages as ordinates, however, the formula won't hold. – EdM Apr 11 '16 at 12:07
• Let's not worry about the percentages, they are just a scaling of the original data. So you say $\frac{1}{m} \approx 0$. Ok, it makes sense. What about the last term of the root ? I can't figure out how I should calculate it. – Rififi Apr 11 '16 at 12:24
• Your full data are classic sigmoid curves with under- and overshoot. A more formal analysis based on a generalized model of the raw data that resulted in these percentages would be ideal. If all you care about is the $x$ value at 50% response, then restricting to the middle quasi-linear part of the curve may be good enough. Technically, your having pre-selected the "linear part of the curve" means that p-values aren't going to be reliable. You should check out different such selections, as variability from that choice may overwhelm the "linear"-regression error. – EdM Apr 11 '16 at 19:46

There is a relatively simple resolution of this problem: compute a “fiducial limit” based on “inverse regression” [Draper & Smith 1981]. The idea is to create confidence envelopes for the true line and then find the range of $$X$$ values where these envelopes enclose the target response.

After introducing some notation (intended to match that in Draper & Smith), this answer performs a preliminary analysis of the situation, illustrates the idea with a plot of simulated data, and presents the formulas. It concludes with a brief discussion (in which a simple approximation is presented) and a reference to the principal source of this solution, Draper & Smith's regression textbook.

(The source of this answer is a report I wrote years ago concerning ongoing monitoring of concentrations in the environment: the $$X_i$$ were time and the $$Y_i$$ were log concentrations. The problems of (a) monitoring to determine when a value will reach a predetermined target and (b) calibration of measurement systems--where the $$X_i$$ are known values and the $$Y_i$$ are the instrument's responses--are the two situations in which I have found this procedure to be most useful.)

Let's establish notation. The data are $$(X_i, Y_i)$$, $$i=1, 2, \ldots, n$$. The model is

$$Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i$$

for unknown parameters $$\beta_0$$ (the intercept) and $$\beta_1$$ (the slope) and independent Normal, zero-mean variates $$\varepsilon_i$$ with common (unknown) variance $$\sigma^2$$. Ordinary Least Squares regression obtains estimates $$b_0$$, $$b_1$$, and $$s$$ of the unknowns $$\beta_0$$, $$\beta_1$$, and $$\sigma$$. The calculations that enter into those estimates include the means $$\bar X$$ and $$\bar Y$$ as well as the sum of squared deviations of the $$X_i$$,

$$S_{XX} = \sum_{i=1}^n (X_i - \bar{X})^2.$$

### Analysis

To begin the analysis, note that the regression line necessarily passes through the point of averages $$(\bar{X}, \bar{Y})$$, signifying an average response $$\bar Y$$ attained at the average ordinate $$\bar X$$. Moreover, the abscissa $$\bar Y$$ is Normally distributed, uncorrelated with the estimated slope $$b_1$$, and has a standard error that decreases to zero as the amount of data increases. The value of $$X$$ for any given $$Y_0$$ can be estimated by starting here and extrapolating, yielding an estimate of

$$\hat{X}_0 = (\bar{Y} - Y_0)/b_1 + \bar X.$$

The second step is to note that for any value $$X$$ we can compute an upper confidence limit for the fitted response at $$X$$. The need for a confidence limit arises from uncertainty about the values of the coefficients $$\beta_0$$ and $$\beta_1$$: we are not exactly sure of the true intercept and true slope, so the true line really could lie within a range of possible lines. The fitted response at $$X$$ can be written

$$\hat{Y}(X) = \bar{Y} - b_1(X - \bar{X})$$

and the standard error of that fitted value equals

$$\operatorname{se}(\hat{Y}(X)) = s\left(\frac{1}{n} + \frac{(X - \bar{X})^2}{S_{XX}}\right)^{1/2}.$$

The fitted value is Normally distributed, whence an upper confidence limit of confidence $$1 - \alpha$$ can be constructed of the form

$$\operatorname{UCL}(X) = \hat{Y}(X) + t(n-2, \alpha) \operatorname{se}(\hat{Y}(X))$$

and a lower confidence limit (LCL) is constructed analogously. (As usual, $$t$$ refers to percentage points of a Student $$t$$ distribution.) As $$X$$ varies, the UCL and LCL trace hyperbolic arcs lying above and below the fitted line.

The horizontal axis plots the $$X$$ values while the vertical axis plots the $$Y$$ values. The hyperbolic arcs are shown as green (LCL) and yellow (UCL) curves. The fiducial limits are found by intersecting these arcs with a horizontal line at the height $$Y_0\,$$ designated "Target" in the legend. The resulting UCL is shown with a diamond symbol. This illustration uses simulated data: this allows us to see how additional data might reasonably vary from what the calculations lead us to expect. (The reason why "observed" and "simulated" values have been visually connected is that this shows a plot of concentration vs. time of a presumably continuous process.)

### Solution

To find the “upper fiducial limit,” or “inverse confidence limit for $$X$$ given $$Y_0$$” ([Draper & Smith 1981] section 1.7), find the largest solution $$X$$ of the equation

$$Y_0 = \operatorname{UCL}(X),$$

if such a solution exists. This can be solved with the quadratic formula, giving

$$\operatorname{UCL}(X) = \bar{X} + \frac{D_0 + g\sqrt{D_0^2 + (1-g^2)S_{XX}/n}}{1-g^2}, \tag{1}$$

where

$$D_0 = (\bar Y - Y_0) / b_1$$

is the estimated value of $$X$$ corresponding to $$Y_0$$,

$$g^2 = \frac{t^2 s^2}{b_1^2 S_{XX}}$$

is an auxiliary calculation, and

$$t = t(n-2, \alpha).$$

A lower confidence limit on $$X$$ is obtained by using the negative square root $$–g$$ in $$(1)$$. (These formulas are equivalent to [Draper & Smith] equation 1.7.6. I write $$g^2$$ here in place of their $$g$$. This version is a little easier to compute with.)

### Discussion

Neither confidence limit has to exist. They can be found only when there is confidence that the slope truly is nonzero. Draper & Smith suggest that computing confidence limits for $$X$$ is “not of much practical value” unless $$g^2 < 0.2$$, although they do not provide any justification for such an omnibus statement.

When $$g^2$$ is relatively small, a good approximation is obtained by expanding $$(1)$$ in a power series in its positive square root $$g$$ and stopping after the linear term, yielding

$$\operatorname{UCL}(X) \approx \bar{X} + D_0 + g\sqrt{D_0^2 + S_{XX}/n} + \cdots\tag{2}.$$

Note that $$g^2$$ is small when, relative to the estimated variance $$s^2$$, the estimated coefficient $$b_1$$ is large, the variance of the $$X_i$$ (that is, $$S_{XX}/n$$) is large, and $$t$$ is small (that is, extremely high confidence is not required). In short, any combination of a large absolute slope, wide spread in the $$X_i$$, large amounts of data, relatively small variation around a linear curve, and/or modest confidence needs will assure the approximation $$(2)$$ is a good one. Also note that as additional data are collected throughout a range of $$X$$ values that cover the confidence limit, $$\operatorname{UCL}(X)$$ converges to $$\bar{X}+D_0$$ , the estimated value, as one would expect of a genuine confidence limit.

### References

Draper, NR and H Smith, 1981: Applied Regression Analysis, Second Edition. John Wiley & Sons, New York.