4
$\begingroup$

I implemented first order deming regression on an array of x and y values. I tried to calculate the confidence interval, which in my understanding should be two values (upper and lower) for slope and two values for intercept. When I plot these two values they should be two straight lines too

enter image description here

However, I have seen in few analyses showing confidence for linear regression as a curve? (example in picture below) enter image description here.

I do not understand where this comes from, what does this represents and how to implement in in a first order regression. So if anyone can guide me a place to start to understand about this would be a great help.

$\endgroup$
9
  • $\begingroup$ The second picture shows simultaneous confidence bands when using OLS.. The first uses Deming regression which minimizes distance based on an error in variables model (in this case for X). But here are a lot to question about comparing the graphs. (1) Different scales, (2) different scatter plots with different number of data points exhibited and (3) very different slopes. $\endgroup$ Commented Jan 17, 2017 at 14:16
  • $\begingroup$ @MichaelChernick I got the impression that the first plot was OLS with the questioner's manual (maybe mistaken) computation of the error bands -- or is first order (I read this as not polynomial) another name for Deming regression? $\endgroup$
    – jwimberley
    Commented Jan 17, 2017 at 14:25
  • 1
    $\begingroup$ Deming regression refers to error in variables not a particular form like a polynomial in the predictor variable. There is a lot to be puzzled about. Hopefully the OP will be able to straighten this out. $\endgroup$ Commented Jan 17, 2017 at 14:31
  • $\begingroup$ Ah, I missed the reference to Deming regression in the question. $\endgroup$
    – jwimberley
    Commented Jan 17, 2017 at 14:57
  • $\begingroup$ @jwimberley Deming regression is just one example. Say for the first graph is also OLS; you will still get 2 straight lines for CI out of the upper and lower boundaries. My question here is specific to why the second graph is curved CI and what does it represents? $\endgroup$
    – Sharah
    Commented Jan 17, 2017 at 15:53

2 Answers 2

4
$\begingroup$

In OLS, one cannot treat the effects of the slope and intercept terms separately. A prediction $$ \hat y = \beta_0 + \beta_1 x $$ is a function of both $\beta_0$ and $\beta$ simultaneously. A proper way to find a confidence interval on this quantity is standard propagation of uncertainty. First calculate the gradient of $\hat y$ w.r.t. the two parameters: $$ \nabla_\beta \hat y = \left[ 1, x \right]^T $$ The uncertainty on $\hat y$ is given by $$ \sigma^2_{\hat y} = \left(\nabla_\beta \hat y\right)^T \Sigma_{\beta} \nabla_\beta \hat y $$ where $\Sigma_{\beta}$ is the covariance matrix of $\beta_0$ and $\beta_1$. This is equal to $$ \Sigma_\beta = \left( \begin{array}{cc} \sigma^2_0 & \rho \sigma_0 \sigma_1 \\ \rho \sigma_0 \sigma_1 & \sigma_1^2 \end{array} \right) $$ where $\rho$ is the correlation between $\beta_0$ and $\beta_1$. From here you can easily calculate the variance $\sigma^2_{\hat y}$ and set a confidence interval. You'll find that it is a quadratic function of $x$. Most statistics packages in R and Python have functions that will do this for you automatically (hence the curved lines that your question inquires about).

EDIT Regarding confidence intervals in Deming regression: this is a separate issue that my answer does not address.

$\endgroup$
1
  • $\begingroup$ The red line indicates a linear relationship and the curve just represents simultaneous confidence bounds on the regression function when OLS is used. $\endgroup$ Commented Jan 17, 2017 at 14:34
1
$\begingroup$

The Confidence Intervals for traditional Linear Regression (the curved lines) are defined as: $$CI\left(\hat{Y}\right)=\pm t_{\alpha,df}\times s_{res} \times \sqrt{\frac{1}{n}+\frac{\left(x-\overline{x}\right)^2}{S_{xx}}}$$ where $t$ is the students $t$ distribution, $\alpha$ is your desired confidence interval, $df$ are the degrees of freedom for your model, $s_{res}$ is the standard deviation in the sum of the squares of the residuals, $n$ is the number of samples, $x$ is the individual data point, $\overline{x}$ is the sample mean, and $S_{xx} is the sum of the squares of the deviations.

The various variables each impact the final line and adjust the change in curvature.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.