0
$\begingroup$

Suppose you have fitted an linear regression to an old dataset. But You only have the coefficients and not the dataset anymore. How do you fit a linear regression model to a new dataset?

I’m thinking of just reproducing an old dataset and run the regression on it. So that means if for any two datasets $(X_1, Y_1)$ and $(X_2, Y_2)$ of same $\beta = (X^*X)^{-1}X^*Y$, their union to a third dataset $(X_3, Y_3)$ have the same parameter, then my approach is valid. But it seems highly unlikely.

I’m also thinking of running stochastic gradient descent, and viewing the old coefficients as the point derived by gradient descent on the previous points. But I’m not sure of the step size in this case. Is there any clever idea?

Improve this question
$\endgroup$
4
  • $\begingroup$ What’s wrong with the usual $\hat\beta_{ols}=(X^TX)^{-1}X^Ty$ with the new data? Do you mean that you want to fit the regression to the old and new data pooled together (get the answer that you would get easily if you had both data sets)? $\endgroup$
    – Dave
    Commented Jul 2, 2021 at 19:49
  • 1
    $\begingroup$ Yeah I want to fit the regression to the old and new data pooled together despite only having the coefficient of the linear regression model on the the old data $\endgroup$
    – The One
    Commented Jul 2, 2021 at 20:02
  • 2
    $\begingroup$ This program is doomed, because the coefficients alone aren't enough. See stats.stackexchange.com/questions/532820 for essentially the same question -- the comments give some information about which statistics will suffice. There's a huge pitfall that can be appreciated by regressing the dataset $(0,0),(1,2),(2,2),$ storing the coefficients, and contemplating what ought to happen when adjoining the new dataset $(100,-100)$ (draw the scatterplot). $\endgroup$
    – whuber ♦
    Commented Jul 2, 2021 at 21:10
  • 1
    $\begingroup$ @whuber Nice point. It is similar in neural networks, weights aren't enough to continue with the new data training. $\endgroup$
    – patagonicus
    Commented Jul 2, 2021 at 21:35

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.