If $\hat{e}$ are the OLS residuals, what is random in $\hat{\beta}_{OLS}|\hat{e} = e_0$?

Question

Suppose $y \sim N(\mu, \Sigma)$, where $y \in \mathbb{R}^n$. Let $X \in \mathbb{R}^{n \times p}$ denote a full rank design matrix. By ordinary least squares, the residuals are $$\hat{e} = (I - X(X^TX)^{-1}X^T)y$$ Let $u_1 \in \mathbb{R}^p$ denote a unit vector in $\mathbb{R}^p$. Let $\hat{\beta}_1 = u_1^T (X^TX)^{-1}X^Ty$ denote the OLS estimate for the first covariate in $X$. Suppose I'm interested in the condition distribution of $$\hat{\beta}_1 | \hat{e} = e_0$$

My question is, once I condition on a specific set of the residuals $\hat{e} = e_0$, is there still randomness left in $\hat{\beta}_1$? I'm confused about this because $\hat{\beta}_1$ is a function of $y$, so any variability in $\hat{\beta}_1$ stems from the fact that $y$ is random. Since $\hat{e}$ is a function of $y$, does conditioning on a specific value of $\hat{e}$ imply that there's no more variability left in $\hat{\beta}_1$? In other words, is it possible for different $y$ vectors to give rise to the same residuals, $e_0$?

Answer 1

We know that the vector $\begin{bmatrix} (X^TX)^{-1} X^T \\ I - X (X^TX)^{-1}X^T \end{bmatrix}y$ has a multivariate normal distribution, since it's a linear transformation of a normal distribution. Now, in the case of a multivariate normal, we know that dependence is characterized simply by covariance, so let's investigate the covariance between the first and second block of this multivariable normal distribution.

We will use the result that $\mathrm{cov}(Ay, By) = A \mathrm{cov}(y) B^T$ for conformable matrices $A,B$ and assume that $\mathrm{cov}(y) = \sigma^2 I$. Then the covariance is $$(X^TX)^{-1} X^T \left( \sigma^2 I \right) \left( I - X (X^TX)^{-1} X^T \right) = 0.$$ This means that the blocks are uncorrelated and hence independent. Therefore conditioning on the second half of the block does not change the distribution of the first half the block, i.e. $\hat\beta \stackrel{d}{=} \hat\beta \mid \hat{e}$. So the answer to the question of what is random: all of it.

Answer 2

To give an explicit example to illustrate that residuals may remain the same even if the dependent variable changes, consider

x <- rep(1:3,2)
y <- c(x[1:3]+1,x[1:3]+3)
resid(lm(y~x))
plot(x,y,ylim=c(1,8))
abline(lm(y~x))

y2 <- c(1,3,5,3,5,7)
points(x,y2, col="red")
abline(lm(y2~x), col="red")
resid(lm(y~x))