2
$\begingroup$

I have a multiple regression (e.g. $y_i = \beta_1 x_{i1} + \beta_2 x_{i2} + \cdots + \beta_p x_{ip} + \varepsilon_i$), in which I want to demonstrate that $\beta_1 > \beta_2$. I have the full population's data available for the analysis, but because of the size of the data (in order of tens of billions of observations) I need to resort to downsampling in order to run the inferential analysis.

When I run the regressions on a sample and run the Wald test on the coefficients, the difference between $\beta_1$ and $\beta_2$ is rarely significant. But when I resample I see that $\beta_1$ is larger than $\beta_2$ almost in every instance.

So, to satisfy my curiosity I just made 100 different samples out of the population data and ran the regressions 100 times. Then I ran a t-test to compare the means of the two coefficient estimates for these 100 regressions, and the difference is highly significant!

But does that constitute a valid argument for the inequality of those coefficients? Is there any proper method that allows me to "stack" the regressions to benefit from the added information?

Improve this question
$\endgroup$
2
  • $\begingroup$ This is an interesting question I should say. I'm not surprise that Wald test is not significant. A rule of thumb for these scenarios (including LRT) is to do permutation or bootstrapping. $\endgroup$
    – NULL
    Commented Aug 14, 2017 at 12:24
  • $\begingroup$ @NULL: Do you mean bootstrapping the entire regression models, or the mean coefficient difference test? I have already calculated the mean coefficient difference CIs at 99.9% and the CI does not include zero. $\endgroup$
    – retrography
    Commented Aug 14, 2017 at 14:40

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.