# Confidence interval for first order linear regression

I implemented first order deming regression on an array of x and y values. I tried to calculate the confidence interval, which in my understanding should be two values (upper and lower) for slope and two values for intercept. When I plot these two values they should be two straight lines too However, I have seen in few analyses showing confidence for linear regression as a curve? (example in picture below) .

I do not understand where this comes from, what does this represents and how to implement in in a first order regression. So if anyone can guide me a place to start to understand about this would be a great help.

• The second picture shows simultaneous confidence bands when using OLS.. The first uses Deming regression which minimizes distance based on an error in variables model (in this case for X). But here are a lot to question about comparing the graphs. (1) Different scales, (2) different scatter plots with different number of data points exhibited and (3) very different slopes. Jan 17, 2017 at 14:16
• @MichaelChernick I got the impression that the first plot was OLS with the questioner's manual (maybe mistaken) computation of the error bands -- or is first order (I read this as not polynomial) another name for Deming regression? Jan 17, 2017 at 14:25
• Deming regression refers to error in variables not a particular form like a polynomial in the predictor variable. There is a lot to be puzzled about. Hopefully the OP will be able to straighten this out. Jan 17, 2017 at 14:31
• Ah, I missed the reference to Deming regression in the question. Jan 17, 2017 at 14:57
• @jwimberley Deming regression is just one example. Say for the first graph is also OLS; you will still get 2 straight lines for CI out of the upper and lower boundaries. My question here is specific to why the second graph is curved CI and what does it represents? Jan 17, 2017 at 15:53

In OLS, one cannot treat the effects of the slope and intercept terms separately. A prediction $$\hat y = \beta_0 + \beta_1 x$$ is a function of both $\beta_0$ and $\beta$ simultaneously. A proper way to find a confidence interval on this quantity is standard propagation of uncertainty. First calculate the gradient of $\hat y$ w.r.t. the two parameters: $$\nabla_\beta \hat y = \left[ 1, x \right]^T$$ The uncertainty on $\hat y$ is given by $$\sigma^2_{\hat y} = \left(\nabla_\beta \hat y\right)^T \Sigma_{\beta} \nabla_\beta \hat y$$ where $\Sigma_{\beta}$ is the covariance matrix of $\beta_0$ and $\beta_1$. This is equal to $$\Sigma_\beta = \left( \begin{array}{cc} \sigma^2_0 & \rho \sigma_0 \sigma_1 \\ \rho \sigma_0 \sigma_1 & \sigma_1^2 \end{array} \right)$$ where $\rho$ is the correlation between $\beta_0$ and $\beta_1$. From here you can easily calculate the variance $\sigma^2_{\hat y}$ and set a confidence interval. You'll find that it is a quadratic function of $x$. Most statistics packages in R and Python have functions that will do this for you automatically (hence the curved lines that your question inquires about).
The Confidence Intervals for traditional Linear Regression (the curved lines) are defined as: $$CI\left(\hat{Y}\right)=\pm t_{\alpha,df}\times s_{res} \times \sqrt{\frac{1}{n}+\frac{\left(x-\overline{x}\right)^2}{S_{xx}}}$$ where $t$ is the students $t$ distribution, $\alpha$ is your desired confidence interval, $df$ are the degrees of freedom for your model, $s_{res}$ is the standard deviation in the sum of the squares of the residuals, $n$ is the number of samples, $x$ is the individual data point, $\overline{x}$ is the sample mean, and \$S_{xx} is the sum of the squares of the deviations.