1
$\begingroup$

This is something that is really causing me a lot of confusion.

Consider the functional form y = A + Bx.

As far as I understood, we could estimate the population parameters A and B with say a and b using least squares linear regression, and calculate the standard error of these respectively. For example, the standard error of the slope would be SQRT(Standard Error^2 / Sum of Squares XX). This is something that could be calculated exactly. This of course assumes that these estimates are normally distributed - but I thought that was the main assumption in linear regression?

My understanding is that by bootstrapping the confidence intervals, we don't need to assume anything about their distribution. But won't the outputs of the bootstrapped confidence intervals be normally distributed, anyways? If it isn't going to be normally distributed, does that violate the normality assumption?

So I guess my questions are:

  1. What is the purpose of bootstrapping?
  2. Is bootstrapping always recommended over the traditional calculation (provided you have the computing power / expertise)? I.e. you would only do the other method, if you don't have the time / skills to apply bootstrapping.

I know it's a really silly question, but the more I read around the subject the more confused that I get.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ No, the assumption of normality is on the residuals, not on the coefficients. You can have coefficients with skew, where the symmetrical CI as estimated exactly would not apply, but you can still have normally distributed residuals. $\endgroup$
    – user2974951
    May 20, 2021 at 10:59
  • $\begingroup$ Thanks for the response. That's a good distinction that I had forgotten, regarding the normality of the residuals. If the coefficients could have skew, is it always recommended then to apply bootstrapping just in case (provided you can)? Secondly, when calculating the prediction / confidence intervals around the line of best fit, would that also be skewed if the parameters were skewed? My guess is yes? $\endgroup$
    – TheFriendlyAsker
    May 20, 2021 at 11:03
  • 1
    $\begingroup$ @user What would it mean to have "coefficients with skew"? The coefficients are parameters, not random variables. The estimated coefficients are linear combinations of the data, whence when the errors are iid Normal, the estimates must also have Normal distributions, which are not skewed. One might instead choose to bootstrap CIs when any of the usual assumptions are violated; e.g. the errors might have skewed distributions; or they might not be independent; or they might not be homoscedastic; or they might not even have constantly zero expectations. $\endgroup$
    – whuber ♦
    May 20, 2021 at 12:17
  • $\begingroup$ Actually, the given standard error formula is correct under non-normality, provided all the rest of the assumptions (linearity, constant variance, and independence) are all correct. If those other assumptions are violated, then the formula is incorrect. If you bootstrap in such a way to mimic the correct data-generating process, then you get better standard error estimates. That is a big "if", however, as there are many ways to bootstrap, but only a few ways to do it correctly for the given data generating process. $\endgroup$
    – BigBendRegion
    May 20, 2021 at 14:40
  • $\begingroup$ The main purpose of bootstrapping under non-normality, then, when all other assumptions are correct, is to get better critical values than $t_{1-\alpha/2, n-2}$, not to get better standard errors. (Assuming, again, that you choose the right bootstrap.) $\endgroup$
    – BigBendRegion
    May 20, 2021 at 14:43

1 Answer 1

Reset to default
1
$\begingroup$

I decided to make my comments a direct answer:

The given standard error formula is correct even under non-normality, provided all the rest of the assumptions (linearity, constant variance, and independence) are all correct. If those other assumptions are violated, then the given formula is incorrect. If you bootstrap in such a way to mimic the correct data-generating process, then you get better standard error estimates. That is a big "if", however, as there are many ways to bootstrap, but only a few ways to do it correctly for the given data generating process.

The main purpose of bootstrapping under non-normality, then, when all other assumptions are correct, is to get better (possibly asymmetric) critical values than $\pm t_{1−\alpha/2,n−2}$, not to get better standard errors. (Assuming, again, that you choose the right bootstrap.) These in turn can lead to better coverage rates (i.e., closer to $100(1-\alpha)$%) for the confidence intervals.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.