Effects of Oversampling on Linear Regression We often see that oversampling is a useful technique to combat skewed classes. But there is not much discussion about linear regression. If we just duplicate the original data, and train the linear regression model on the new larger dataset, are there any significant good/bad effects of it on the test set predictions?
In particular, will it cause the coefficient to change?
Advice or link to papers is appreciated.
 A: There is actually quite an expansive and rich literature on oversampling and linear regression. The general survey methods literature has discussed and explored many types of estimators. For starters, you might reference "Complex Surveys: A Guide To Analysis Using R" by Lumley.
Duplicating data is not oversampling. The point of oversampling is to increase generalizability (valid inference and predictions with better external validity). Copy-pasting data actually decreases these qualities significantly.
Skewness is a property of continuous variables. For categorical covariates, we might speak of the balance of their frequencies... but even then the goal of oversampling isn't to create balance. We just want to have adequate precision in the subgroups defined by the stratification variables.
For instance, suppose I want to estimate the efficacy of an Alzheimer's Disease preventative medication which I believe interacts with the apoe alleles that are overexpressed in American Indian (AIAN). In a sample of 500 at-risk US-based patients, I expect fewer than 5 AIANs. I might oversample to adequately estimate the AIAN-specific effect, then use marginal standardization to reweight the race-specific effects according to the known US-based distribution of race.
