We often see that oversampling is a useful technique to combat skewed classes. But there is not much discussion about linear regression. If we just duplicate the original data, and train the linear regression model on the new larger dataset, are there any significant good/bad effects of it on the test set predictions?

In particular, will it cause the coefficient to change?

Advice or link to papers is appreciated.

  • 3
    $\begingroup$ The coefficients will be identical to those from the non-duplicated data. Ignoring standard errors, the predictions will be identical too. $\endgroup$ – Noah Jun 20 '19 at 17:04
  • 1
    $\begingroup$ Statistical analysis is used to analyze data, and not to generate data. "duplicate the original data" is totally wrong and very dangerous idea. $\endgroup$ – user158565 Jun 21 '19 at 0:32

There is actually quite an expansive and rich literature on oversampling and linear regression. The general survey methods literature has discussed and explored many types of estimators. For starters, you might reference "Complex Surveys: A Guide To Analysis Using R" by Lumley.

Duplicating data is not oversampling. The point of oversampling is to increase generalizability (valid inference and predictions with better external validity). Copy-pasting data actually decreases these qualities significantly.

Skewness is a property of continuous variables. For categorical covariates, we might speak of the balance of their frequencies... but even then the goal of oversampling isn't to create balance. We just want to have adequate precision in the subgroups defined by the stratification variables.

For instance, suppose I want to estimate the efficacy of an Alzheimer's Disease preventative medication which I believe interacts with the apoe alleles that are overexpressed in American Indian (AIAN). In a sample of 500 at-risk US-based patients, I expect fewer than 5 AIANs. I might oversample to adequately estimate the AIAN-specific effect, then use marginal standardization to reweight the race-specific effects according to the known US-based distribution of race.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.