Let's say I have the following multivariable linear regression: y = A x + b, x being a vector of many variables.

Is it possible to remove a specific variable x with non-null coefficient in a way the resulting model will be as it had never learned that variable in the first place?

I want to be agnostic in relation to the training procedure, so I don't want to retrain the model with one less variable.

My suspicion is that if the mean of the variable is zero, just removing it would work, otherwise the bias b would need to be corrected somehow.

  • $\begingroup$ One regression does not remember what you did previously. So, just exclude what you want to exclude. $\endgroup$ – Nick Cox Dec 9 '13 at 13:18
  • $\begingroup$ I mean I want to exclude the variable without having to do the training again. $\endgroup$ – Pedro Tabacof Dec 9 '13 at 13:35
  • 2
    $\begingroup$ What "training" is involved here? You don't give any details of what you did except "regression". If you are using some flavour of machine learning, you should explain what that is. $\endgroup$ – Nick Cox Dec 9 '13 at 13:37
  • $\begingroup$ In my case I'm using Lasso, but I wanted an agnostic answer in relation to the training procedure. $\endgroup$ – Pedro Tabacof Dec 9 '13 at 13:41
  • 2
    $\begingroup$ It should be stated or at the very least implied by every regression text. The converse is that if you add a new predictor, expect coefficients to change unless the new predictor is uncorrelated with all the others. Try it with data.... (You seem to be trying to learn statistics by searching the internet. You need to work through reliable textbooks, not browse for what you think you want.) $\endgroup$ – Nick Cox Dec 9 '13 at 15:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.