0
$\begingroup$

My manager is interested in using Adjusted r-squared to evaluate models and want to know how much does each predictor contribute to the adjusted r-squared to the final model.

However, due to an imbalanced data set along with a relatively small sample size, we bootstrapped our data in order to rebalance our data that is more reflective of the population. For the sake of conversation, let's say we ran 10 bootstraps.

For each bootstrapped sample, we did step-wise variable selection to determine what are the best predictors for each bootstrapped sample, and ran a linear regression for each bootstrap sample. Thus, the same predictors were not used for each bootstrapped sample (due to variable selection).

The problem I'm facing is how do I pool the results for my manager's needs? Sure I can still calculate the standard errors and the average coefficients for each predictor that I use, but then how do I show how much each "significant" predictor is contributing to the adjusted r-squared?

Do I average the adjusted r-squared across all 10 bootstrapped samples and say that is the final adjusted r-squared. And then report the predictors that are "consistently" significant across all bootstrap samples, average each of their coefficients, and do a proportion calculation to see how much they are contributing to the adjusted r-squared? (Note, all predictors were standardized before hand).

$\endgroup$
  • $\begingroup$ Bootstrapping is intended to resample from the original sample using the same sample size. It is not used to adjust or :balance the data". $\endgroup$ – Michael Chernick May 11 '18 at 23:25
  • $\begingroup$ @MichaelChernick, yes, that is true. The one concern I have is that if I just rebalance my dataset one time, the final dataset that I have may not be representative of the population. So what I did was I resampled with replacement on both groups, but did it in a way that the ratio of the two groups is more representative of the population. Is there merit to rebalancing the population numerous times (similar to bootstrapping) so that you have multiple datasets that may be more representative of the population? Or is this something dangerous that I am doing? $\endgroup$ – Kevin Sun May 11 '18 at 23:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.