In OLS, does the uncorrelatedness between regressors and residuals require a constant?

It shows how to obtain the OLS estimator and its properties.

It is said that from the normal equations we obtain $$X' e = 0$$.

Where $$X$$ is the design matrix and $$e$$ is the vector of residuals.

Then, at page 4, it claims the following property: "The observed values of X are uncorrelated with the residuals."

Proof: Indicating the regressors by $$x_k$$, $$X' e = 0$$ implies $$x'_k e = 0$$.

So far so good.

Then it goes on to say "In other words, each regressor has zero sample correlation with the residuals."

I didn't understand that.

$$r_{x_pe} = \frac{\sum_{i=1}^n x_{pi} e_i - n \bar{x_p} \bar{e}}{n s^{'}_{x_p} s^{'}_e}$$

We have proven that $$\sum_{i=1}^n x_{pi} e_i = 0$$. But then there is another term at the numerator.

Except if the mean of the residuals equal 0. That is, if $$\bar{e}=0$$.

But that would require the constant, as shown in point 3 of page 4.

• Commented Jan 17, 2023 at 2:50

You are right.

Maybe because most regressions do contain a constant, the property $$X'e=0$$ (often called, more precisely, "orthogonality") and the terminology "uncorrelatedness" are often used interchangeably, when they do amount to the same thing only if the regression contains a constant (or, more precisely, if the residuals have mean zero, which can also be the case if the regressors can be linearly combined into a constant, say with an exhaustive set of dummies).

A little numerical illustration:

n <- 10
y <- rnorm(n)
x <- rnorm(n)

regwcst <- lm(y~x)
regwocst <- lm(y~x-1)
d1 <- c(rep(1,5), rep(0,5)) # two exhaustive dummies
d2 <- 1-d1
regwdumm <- lm(y~x-1+d1+d2)

> crossprod(x, resid(regwcst))  # all numerically zero
[,1]
[1,] -2.081668e-17

> crossprod(x, resid(regwdumm))
[,1]
[1,] -1.249001e-16

> crossprod(x, resid(regwocst))
[,1]
[1,] 1.804112e-16

> cor(x, resid(regwcst))        # numerically zero
[1] -2.721791e-17

> cor(x, resid(regwocst))       # not numerically zero
[1] 0.01718539

> cor(x, resid(regwdumm))       # numerically zero