3
$\begingroup$

This is a follow up question to the question I've posted here.

Suppose $Y \sim N(X\beta, \sigma^2I)$, where $y \in \mathbb{R}^n$. Let $X \in \mathbb{R}^{n \times p}$ denote a full rank design matrix. By ordinary least squares, the residuals are $$\hat{e} = (I - X(X^TX)^{-1}X^T)Y$$ Let $u_1 \in \mathbb{R}^p$ denote a unit vector in $\mathbb{R}^p$. Let $\hat{\beta}_1 = u_1^T (X^TX)^{-1}X^TY$ denote the OLS estimate for the first covariate in $X$. Suppose I'm interested in the conditional distribution of $$\hat{\beta}_1 | f(\hat{e}) < \hat{\beta}_1 < f^*(\hat{e}), \hat{e} = e_0$$

where $f(e_0)$ and $f^*(e_0)$ are some functions of $e_0.$ From the answer to my previous post, $\hat{\beta}_1$ and residuals $\hat{e}$ are independent, so I can drop $\hat{e} = e_0$ from the conditioning to obtain

$$\hat{\beta}_1 | f(e_0) < \hat{\beta}_1 < f^*(e_0)$$

Since $\hat{\beta}_1$ is normal, then truncating $\hat{\beta}_1$ between some fixed quantities $f(e_0)$ and $f^*(e_0)$ leads to a truncated normal. Therefore, $\hat{\beta}_1|f(e_0) < \hat{\beta}_1 < f^*(e_0)$ is a truncated normal with lower truncation limit $f(e_0)$ and upper truncation limit $f^*(e_0)$

My question is, once I condition on $\hat{e} = e_0,$ is there still randomness left in $\hat{\beta}_1$? Based on my previous post, I learned that $$[\hat{\beta}_1|\hat{e} = e_0] \overset{d}{=} \hat{\beta}_1$$

Is the same true in this case? i.e.,

$$[\hat{\beta}_1 | f(e_0) < \hat{\beta}_1 < f^*(e_0)]\overset{d?}{=} \hat{\beta}_1$$

The reason I feel that this is not true is because changing the value of $e_0$ would lead to a different set of truncation limits on $\hat{\beta}_1$. On the other hand, if $\hat{\beta}_1$ is independent of $\hat{e}$, then it's independent from any functions of $\hat{e}$. This might suggest that $[\hat{\beta}_1 | f(e_0) < \hat{\beta}_1 < f^*(e_0)]\overset{d}{=} \hat{\beta}_1$. I am quite confused and would greatly appreciate any insight on this.

Improve this question
$\endgroup$
6
  • $\begingroup$ I'm a little confused by your notation. How does your notation $y\sim N(\mu,\sigma^2 I)$ relate to the rest of the problem. Based on the rest of the problem, I would have guessed you need something like $Y|X\sim N(X\beta,\sigma^2 I)$? Also, what is $f$? Is it assumed known but arbitrary? Does it need to be invertible? And it sounds like it maps from $\mathbb{R}^n$ to $\mathbb{R}$? $\endgroup$
    – psboonstra
    Commented Apr 1, 2021 at 20:33
  • $\begingroup$ Yes, technically $y|X \sim N(X\beta, \sigma^2I)$. $f$ is arbitrary and maps from $\mathbb{R}^n$ to $\mathbb{R}$. $\endgroup$
    – Adrian
    Commented Apr 2, 2021 at 3:47
  • $\begingroup$ Thanks. another question. I see now that my first comment may have misled you, notationally speaking, because you are using $\beta$ in a different sense than I assumed. I had interpreted $\beta_1$ as the regression coefficient corresponding to the first predictor, but you are writing $\beta_1$ as the fitted value of the outcome given $X=u_1$ for some covariate pattern $u_1$. In other words, what you are calling $\hat\beta_1$, I would write as $\hat Y|X=u_1$, or $\hat Y(u_1)$. Is my understanding correct? $\endgroup$
    – psboonstra
    Commented Apr 2, 2021 at 18:42
  • $\begingroup$ $\beta_1$ is the coefficient corresponding to the first covariate. $u_1$ is a unit vector with 1 in the first position and 0 everywhere else. So $\beta_1 = u_1^T\beta$, where $\beta_1$ is a scalar, and $\beta$ is the entire coefficient vector of length $p$. $\endgroup$
    – Adrian
    Commented Apr 2, 2021 at 20:21
  • $\begingroup$ Ok. I understand a unit vector to be any vector with length 1, and I was confused by your stating that $u_1 \in \mathbb{R}^p$ instead of just saying $u_1=\{1, 0, \ldots, 0\}$. $\endgroup$
    – psboonstra
    Commented Apr 2, 2021 at 20:43

1 Answer 1

Reset to default
2
+50
$\begingroup$

Knowing that $\hat\beta_1$ is statistically independent from the residual vector $\hat e$ is not relevant to the question. Your question comes down to: is it true that $[\hat{\beta}_1 | c_l < \hat{\beta}_1 < c_u]\overset{d}{=}\hat{\beta}_1$ for some arbitrary constants $c_l$ and $c_u$ where $c_l<c_u$? The answer is 'no: they are not equal in distribution'. As a counterexample, assuming that $\hat\beta_1$ is normally distributed, $\Pr(\hat\beta_1 < c_l)>0$ but $\Pr(\hat\beta_1 < c_l | c_l < \hat{\beta}_1 < c_u)=0$.

Addendum To your comment, On the other hand, if $\hat\beta_1$ is independent of $\hat e$ , then it's independent from any functions of $\hat e$, that only applies to functions of $\hat e$ alone. You are proposing to condition on a random variable that is a function of both $\hat e$ and $\hat \beta_1$, which is an entirely different kind of dependence. Altogether.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.