Return to Answer

The errors of the variables may be correlated leading to very large errors in some coefficient when they strongly correlate with others. The matrix $(X^TX)^{−1}$ describes this correlation.

###Error in the regression line

The image below shows intuitively how this changes when adding other regressors.

The intercept is the point where a regression line crosses $x=0$.

• On the left the error of the intercept is the error of the mean of the population.
• On the right the error of the intercept is the error of the regression line intercept.

Confidence regions for correlated parameters

The next image displays the confidence regions (contrasting with confidence intervals) of the above regression in a 2-D plot. Here it takes into account the correlation between the parameters.

The ellipse shows the confidence region which is a based on a multivariate distribution of the slope and intercept which may be related via a correlation matrix. For illustration an alternative type of region is also show. This is depicted by the box which is based on two single variate distributions assuming independence (now the confidence for the single variables is $\sqrt{0.95}$).

By changing the model from $y = a + bx$ to a shifted model $y = a + b(x-35.5)$ we see that the correlation between the slope and intercept changes. Now the "intercept" coincides with the standard error of the line around the point $x=35.5$ which you see in the image above is smaller.

#used model and data
set.seed(1)

xt <- seq(0,40,0.1)
x <- c(1:10)+30
y <- 10+0.5*x+rnorm(10,0,3)

The errors of the variables may be correlated leading to very large errors in some coefficient when they strongly correlate with others. The matrix $(X^TX)^{−1}$ describes this correlation.

Error in the regression line

The image below shows intuitively how this changes when adding other regressors.

The intercept is the point where a regression line crosses $x=0$.

• On the left the error of the intercept is the error of the mean of the population.
• On the right the error of the intercept is the error of the regression line intercept.

Confidence regions for correlated parameters

The next image displays the confidence regions (contrasting with confidence intervals) of the above regression in a 2-D plot. Here it takes into account the correlation between the parameters.

The ellipse shows the confidence region which is a based on a multivariate distribution of the slope and intercept which may be related via a correlation matrix. For illustration an alternative type of region is also show. This is depicted by the box which is based on two single variate distributions assuming independence (now the confidence for the single variables is $\sqrt{0.95}$).

By changing the model from $y = a + bx$ to a shifted model $y = a + b(x-35.5)$ we see that the correlation between the slope and intercept changes. Now the "intercept" coincides with the standard error of the line around the point $x=35.5$ which you see in the image above is smaller.

#used model and data
set.seed(1)

xt <- seq(0,40,0.1)
x <- c(1:10)+30
y <- 10+0.5*x+rnorm(10,0,3)

The errors of the variables may be correlated leading to very large errors in some coefficient when they strongly correlate with others. The matrix $(X^TX)^{−1}$ describes this correlation.

###Error in the regression line

The image below shows intuitively how this changes when adding other regressors.

• On the left the intercept is the error of the mean of the population.
• On the right the intercept is the error of the regression line intercept.

Confidence regions for correlated parameters

The next image displays the confidence regions (contrasting with confidence intervals) of the above regression in a 2-D plot. Here it takes into account the correlation between the parameters.

The ellipse shows the confidence region which is a based on a multivariate distribution of the slope and intercept which may be related via a correlation matrix. For illustration an alternative type of region is also show. This is depicted by the box which is based on two single variate distributions assuming independence (now the confidence for the single variables is $\sqrt{0.95}$).

By changing the model from $y = a + bx$ to a shifted model $y = a + b(x-35.5)$ we see that the correlation between the slope and intercept changes. Now the "intercept" coincides with the standard error of the line around the point $x=35.5$ which you see in the image above is smaller.

#used model and data
set.seed(1)

xt <- seq(0,40,0.1)
x <- c(1:10)+30
y <- 10+0.5*x+rnorm(10,0,3)

The errors of the variables may be correlated leading to very large errors in some coefficient when they strongly correlate with others. The matrix $(X^TX)^{−1}$ describes this correlation.

###Error in the regression line

The image below shows intuitively how this changes when adding other regressors.

The intercept is the point where a regression line crosses $x=0$.

• On the left the error of the intercept is the error of the mean of the population.
• On the right the error of the intercept is the error of the regression line intercept.

Confidence regions for correlated parameters

The next image displays the confidence regions (contrasting with confidence intervals) of the above regression in a 2-D plot. Here it takes into account the correlation between the parameters.

The ellipse shows the confidence region which is a based on a multivariate distribution of the slope and intercept which may be related via a correlation matrix. For illustration an alternative type of region is also show. This is depicted by the box which is based on two single variate distributions assuming independence (now the confidence for the single variables is $\sqrt{0.95}$).

By changing the model from $y = a + bx$ to a shifted model $y = a + b(x-35.5)$ we see that the correlation between the slope and intercept changes. Now the "intercept" coincides with the standard error of the line around the point $x=35.5$ which you see in the image above is smaller.

#used model and data
set.seed(1)

xt <- seq(0,40,0.1)
x <- c(1:10)+30
y <- 10+0.5*x+rnorm(10,0,3)

The errors of the variables may be correlated leading to very large errors in some coefficient when they strongly correlate with others. The matrix $(X^TX)^{−1}$ describes this correlation.

The image below shows intuitively how this changes when adding other regressors.

• On the left the intercept is the error of the mean of the population.
• On the right the intercept is the error of the regression line intercept.

# graphics settings
par(mar=c(4.3,4,2.1,0.7))
layout(matrix(c(1:2),1,byrow=TRUE))

# model + error 
xt <- seq(0,40,0.1)
x <- c(1:10)+30
y <- 10+0.5*x+rnorm(10,0,3)

# model intercept
m1 <- lm(y~1)
summary(m1)

# model intercept + slope
m2 <- lm(y~1+x)
summary(m2)

# condidence intervals of curves
newdata <- as.data.frame(list(x=xt,y=rep(0,401)))
p1 <- predict(m1, newdata= newdata, interval = "confidence" )
p2 <- predict(m2, newdata= newdata, interval = "confidence" )

# plotting
plot(x,y,xlim=c(0,40),ylim=c(0,40))
lines(xt,p1[,1])
polygon(c(rev(xt), xt),
        c(rev(p1[,2]), p1[,3]),
        col = rgb(0,0,0,0.1), border = NA)
title("error of the mean")

plot(x,y,xlim=c(0,40),ylim=c(0,40))
lines(xt,p2[,1])
polygon( c(rev(xt), xt),
         c(rev(p2[,2]), p2[,3]), 
         col = rgb(0,0,0,0.1), border = NA)
title("error of regression line")

The errors of the variables may be correlated leading to very large errors in some coefficient when they strongly correlate with others. The matrix $(X^TX)^{−1}$ describes this correlation.

###Error in the regression line

The image below shows intuitively how this changes when adding other regressors.

• On the left the intercept is the error of the mean of the population.
• On the right the intercept is the error of the regression line intercept.

Confidence regions for correlated parameters

The next image displays the confidence regions (contrasting with confidence intervals) of the above regression in a 2-D plot. Here it takes into account the correlation between the parameters.

The ellipse shows the confidence region which is a based on a multivariate distribution of the slope and intercept which may be related via a correlation matrix. For illustration an alternative type of region is also show. This is depicted by the box which is based on two single variate distributions assuming independence (now the confidence for the single variables is $\sqrt{0.95}$).

By changing the model from $y = a + bx$ to a shifted model $y = a + b(x-35.5)$ we see that the correlation between the slope and intercept changes. Now the "intercept" coincides with the standard error of the line around the point $x=35.5$ which you see in the image above is smaller.

#used model and data
set.seed(1)

xt <- seq(0,40,0.1)
x <- c(1:10)+30
y <- 10+0.5*x+rnorm(10,0,3)