I am trying to understand the intricacies of Linear Regression (1 predictor, 1 response). I am using Istanbul Stock Exchange dataset (https://www.kaggle.com/uciml/istanbul-stock-exchange) to do the same. I am using the column FTSE as my predictor variable and USD BASED ISE as my response variable.
I am writing my own Python (v 3.5.6) methods for all mathematical computation tasks viz. calculating $\beta$s ($\beta_{0}$ and $\beta_{1}$ in my case), calculating MSE, calculating $R^{2}$ etc. I have successfully calculated $\beta$s and could correctly estimate MSE, $R^{2}$ values.
I am trying to understand Boostrap and use the same to estimate

  1. Confidence intervals for coefficients $\beta_{0}$ and $\beta_{1}$
  2. Calculate t-statistic for $\beta_{0}$ and $\beta_{1}$
  3. Calculate p-value for $\beta_{0}$ and $\beta_{1}$

I am trying to calculate the above mentioned statistics without using any statistical package. I have written a function that takes my input dataset and generates 1,00,000 bootstrap samples out of it. For each of these samples I have obtained $\beta_{0}$ and $\beta_{1}$ values. In a nutshell, I have 1,00,000 $\beta_{0}$ and $\beta_{1}$ values. Using these list of values, how do I calculate statistics mentioned in (1), (2) and (3)


1 Answer 1


There are in general multiple ways to calculate bootstrap intervals, see Wikipedia for an overview. The case of bootstrap-based confidence intervals for a linear regression is described in detail here.

For instance, let $\hat{\beta}_{1(1)}, \ldots, \hat{\beta}_{1(b)}$ be your $b=1,000,000$ bootstrap estimates of $\beta_1$. Then, a $(1-\alpha)$-confidence interval for $\beta_1$ can be defined by the $\alpha/2$ and the $(1-\alpha/2)$ quantiles of your bootstrap estimates $\hat{\beta}_{1(1)}, \ldots, \hat{\beta}_{1(b)}$.

Edit: Provided more details.

  • $\begingroup$ Regarding the p-values, I've seen it described like that before, but other answers tend to point out that quantile-based bootstrapping tends to be supoptimal (see stats.stackexchange.com/a/277391/60613) $\endgroup$
    – Firebug
    Commented Oct 22, 2020 at 14:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.