24
$\begingroup$

I'm running various multiple regression analyses in SPSS 27, and with those that are not bootstrapped, the p-values vary such that I do not find the same p-value twice within a regression (e.g., the p-values will be 0.000012435, 0.0053868, 0.000000013845, and so on). However, I bootstrapped some of these regressions (bca, 500 samples), and all p-values listed under the table that indicates the bootstrapped results are multiples of 0.001996 (e.g., 0.001996, 0.003992, 0.007984). Are these legitimate p-values? Or is this an error on the part of SPSS? In either case, are these p-values "report-able", or should I use the p-values of the non-bootstrapped regression results?

$\endgroup$
0

1 Answer 1

46
$\begingroup$

Suppose you have a regression coefficient estimate of 1.2. To compute its p-value, you need to know the probability of observing a coefficient that large (or larger) under the null hypothesis. To do this, you have to know the null distribution of this coefficient. Bootstrap resampling is one way to estimate this null distribution. For a regression, across your 500 bootstrap samples, you're going to get a distribution of regression coefficients with a mean that will be close to 1.2. Let's say the mean of the bootstrap-sampled coefficients is 1.19. Let's also say that your null hypothesis is that the true value of the coefficient is 0. This means that the null distribution of this coefficient should have a mean of 0. We can make our 500 bootstrap-sampled coefficients have a mean of 0 by simply subtracting their current mean of 1.19. This now allows us to use the bootstrap distribution as an estimate of the null distribution.

Then, to calculate a (two-tailed) p-value, we can simply calculate the proportion of our 500 shifted coefficients whose absolute value is larger than the observed value of 1.2. For instance, if 6 of them are larger, that gives us a p-value of 6/500 = 0.012. Notice that any p-value that we calculate this way will always be some integer number divided by 500. So the only p-values that can come out of this calculation are values that are an integer multiple of 0.002, i.e. 1/500.

The values you got were multiples not of 0.002, but of 0.001996. This turns out to be pretty much exactly equal to 1/501. The reason for this discrepancy of 1, is that the "regular" p-value calculated from a bootstrap has a bias. The regular formula is $\hat{p}=\frac{x}{N}$, where $x$ is the number of bootstrap-sampled coefficients that were larger than your observed value, and $N$ is the number of bootstrap samples. The bias-corrected formula is $\hat{p}=\frac{x+1}{N+1}$. So, any p-value resulting from this formula will be an integer multiple of $\frac{1}{N+1}$, which in your case is 1/501.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.