2
$\begingroup$

Note: This question is analagous to the question I asked here except instead of adding a column, I am removing it.

I am interested in a linear regression on the model;

$Y= X\beta + \epsilon$

And I have computed the OLS estimator $\beta$, $\hat{\beta}$;

$\hat{\beta}=(X'X)^{-1}X'Y$

I realize now though that my design is contained an extra column I did not mean to include. For simplicity say the first column $x_1$ should not have been included. I used the matrix $X=[x_1, \tilde{X}]$ when I should have used $\tilde{X}$.

I no longer have access to my $Y$ data - Is there a way for me to update $\hat{\beta}$ to be based on $\tilde{X}$ rather than $X$ when I don't have access to $Y$ anymore? i.e. can I update the OLS solution when I remove a column from the design matrix?

$\endgroup$
1
  • $\begingroup$ Do you have the errors from the original regression? Those would allow you to reconstruct the original Y data. $\endgroup$ May 7, 2019 at 16:18

1 Answer 1

3
$\begingroup$

I am afraid you are again out of luck.

The simplest case would be where your design matrix $X$ contained the intercept column and exactly one additional column... that just happens to be identical to the response, $X=(1|Y)$. In this case, your OLS estimate will be $\hat{\beta}=(0,1)$.

If you now remove the $Y$ column from $X$, you are left with the intercept column. Updating the model is equivalent to finding the mean of the $Y$, because that is the new OLS estimate for $\beta_0$. But you don't have access to $Y$ any more. Finding the mean of unknown data, if all we know is that the data correlates perfectly with itself, is impossible.

The argument is analogous in higher dimensions.

$\endgroup$
1
  • $\begingroup$ Oh no! Thank you for the answers on both threads! I feared this might be the case! $\endgroup$
    – JDoe2
    May 7, 2019 at 17:36

This site is temporarily in read-only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .