# Assuming $y \sim N(X\beta, \sigma^2I)$, and $e = y - X\hat{\beta}$ are residuals, what does $E(\hat{\beta}|e = e_0)$ mean?

Let $$X$$ be an $$n\times p$$ full-rank matrix of predictors with $$n > p$$, and $$y$$ be the vector of responses. Assume $$y \sim N(X\beta, \sigma^2I)$$, and let $$\hat{\beta}$$ be the least-squares estimator of $$\beta$$. Let $$e = y - X\hat{\beta}$$ the vector of residuals.

Since $$y$$ is normal, then $$\hat{\beta} = (X^TX)^{-1}X^Ty \sim N(\beta, \sigma^2(X^TX)^{-1})$$. And $$E(\hat{\beta}) = \beta$$.

What does it mean to take the conditional expectation given the residuals? i.e., $$E(\hat{\beta}|e = e_0),$$ where "=" is interpreted component-wise.

I have two questions:

1. Intuitively, what exactly does it mean to condition on the residuals $$e = e_0$$? Since residuals $$e = y - X\hat{\beta}$$, does this mean that we're assuming that $$y = y_0$$ and $$\hat{\beta} = \hat{\beta}^*$$ such that $$e_0 = y_0 - X\hat{\beta}^*$$? That is, we are conditioning on a specific vector of realizations for the responses $$y_0$$ and a specific vector of OLS coefficients $$\hat{\beta}^*$$?
2. If the above were true, I thought the probability of a continuous random variable is 0. Therefore, is the probability of $$e = e_0$$ also 0 since $$e$$ are continuous?

## Question 1

If $$y\sim\mathcal{N}(X\beta, \sigma^2I)$$, you can obtain $$e$$ via a linear transformation of $$y$$, $$e=(I-H)y$$, this means $$E$$ is normally distributed with mean $$(I-H)X\beta$$ and covariance $$\sigma^2(I-H)(I-H)^T$$. Now you have the pdf $$f_E(e)$$ and its value at some $$e_0$$.

You need to estimate the joint distribution $$f(\hat{\beta}, E)$$. This can be done through a linear transformation $$A$$ with blocks $$(I-H)$$ and $$(X^TX)^{-1}X^T$$

## Question 2

assuming all variables have pdfs, you obviously have $$P(V=v) = 0$$ for any random variable $$V$$. However as long as you have that the pdf is non-zero $$f_V(v) > 0$$ you can define conditional pdfs as $$f(w\mid v) = \dfrac{f_{W,V}(w,v)}{f_V(v)}$$ and compute expectations using $$E[\phi(W)\mid V=v] = \int\phi(w)f(w\mid v)\mathrm{d}w$$ For example think of $$f(W\mid V=v)$$ as being $$W\sim\mathcal{N}(v,\sigma^2)$$. You have zero probability, but the pdf changes as the variable $$v$$ changes.

• Thanks. So what you're saying is, since $e \sim N(0, \sigma^2(I-H))$, where $H = X(X^TX)^{-1}X^T$, then $P(e = e_0) = 0$ because $e$ is a continuous random variable, correct? Jan 26, 2021 at 13:58
• $P(e=e_0)$ is indeed 0 but thats not a problem, since the pdf $f_E$ is normal and therefore never 0 Jan 26, 2021 at 14:22
• Could you explain what does it intuitively mean to condition on $e = e_0$? i.e., for question 1: "does this mean that we're assuming that $y = y_0$ and $\hat{\beta} = \hat{\beta}^*$ such that $e_0 = y_0 - X\hat{\beta}^*$? That is, we are conditioning on a specific vector of realizations for the responses $y_0$ and a specific vector of OLS coefficients $\hat{\beta}^*$?" Jan 26, 2021 at 14:26