1
$\begingroup$

Note: This question is analagous to the question I asked here except instead of adding a column, I am removing it.

I am interested in a linear regression on the model;

$Y= X\beta + \epsilon$

And I have computed the OLS estimator $\beta$, $\hat{\beta}$;

$\hat{\beta}=(X'X)^{-1}X'Y$

I realize now though that my design is contained an extra column I did not mean to include. For simplicity say the first column $x_1$ should not have been included. I used the matrix $X=[x_1, \tilde{X}]$ when I should have used $\tilde{X}$.

I no longer have access to my $Y$ data - Is there a way for me to update $\hat{\beta}$ to be based on $\tilde{X}$ rather than $X$ when I don't have access to $Y$ anymore? i.e. can I update the OLS solution when I remove a column from the design matrix?

$\endgroup$
  • $\begingroup$ Do you have the errors from the original regression? Those would allow you to reconstruct the original Y data. $\endgroup$ – James Phillips May 7 '19 at 16:18
3
$\begingroup$

I am afraid you are again out of luck.

The simplest case would be where your design matrix $X$ contained the intercept column and exactly one additional column... that just happens to be identical to the response, $X=(1|Y)$. In this case, your OLS estimate will be $\hat{\beta}=(0,1)$.

If you now remove the $Y$ column from $X$, you are left with the intercept column. Updating the model is equivalent to finding the mean of the $Y$, because that is the new OLS estimate for $\beta_0$. But you don't have access to $Y$ any more. Finding the mean of unknown data, if all we know is that the data correlates perfectly with itself, is impossible.

The argument is analogous in higher dimensions.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Oh no! Thank you for the answers on both threads! I feared this might be the case! $\endgroup$ – JDoe2 May 7 '19 at 17:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.