Understanding p value from bootstrapping correlations between pairs of variables

Question

My original dataset is small, 15 observations of variables x and y. I have run Spearmann correlation with and without bootstrapping (B=1000).

I have calculated the bias corrected p value as the number of times the bootstrapped test statistic is greater than or equal to the observed test statistic (+1) dividied by the number of bootstraps (+1)

   P = 1+ #(boot test statistics >= orig test statistic)/B +1

   p=(sum(abs(t_boot) >= abs(t_orig)) +1)/(B+1)

If considering a pair of variables (x and y), then the p value from the calculation above is the likelihood of being able to observe a test statistic greater than or equal to the original test statistic, under the null hypothesis ( that there is no correlation between x and y).

Example result using the above approach (estimate, statistic and p value from orig sample)

    Estimate  Statistic   P.value   Boot P value 
      -0.83    302         0.0054      0.29

So from the boot p value of 0.29 here, it can be inferred that 29%, so 290/1000 bootstraps had a test statistic >= observed, under the null hypothesis. Or could be explained as if there is no correlation between x and y, then we would observe a test statistic of 302 or more by chance, 29% of the time. But how does this affect the interpretation of the original observed correlation between x and y, which was significant, its not intuitive to me. Not fishing for statistical significance, but using this example here for my understanding.

I'm aware of another approach from a post on here regarding p values ( where if feasible for Spearman correlation would use the distribution of (Boot coefficient estimate − original cofficient estimate)/bootstrapped se of coefficient estimate) or ((θ^∗−θ^)/σ^∗) as a reference for comparing against the original value of the t statistic . Correct creation of the null distribution for bootstrapped $p$-values

Any help in interpreting the bootstrap p value in the above example would be appreciated.

Answer 1

With an overall sample size of 15 there will always be limits to how well you can approximate anything via resampling. That said, the bootstrap is not suited to approximate the P-value for $H_0: \rho=0$, because the bootstrap does not generate data under the null hypothesis. It would be more appropriate to estimate a standard error and/or confidence interval from your bootstrap, but again, $n=15$.

If you want to resample a P-value you'll need a permutation test, which works directly under a null hypothesis of exchangeability. Basically, you resample one of the two variables without replacement and allocate them to the fixed original other variable, so each X receives a new Y in every sample. In this scheme you can count how many resamples have a statistic that's larger than the original sample. See also here for more explanation and example code.