Bootstrapped t test in SPSS: Why are some p value cells blank?

Using an independent samples t test, I am comparing the performance of 2 groups on an activity, based on the percentage of items correct (the percentage of total items correct, as well as percentage of items correct for four sub-domains of the activity). The n of one group is 23, and the n of the other is 34.

Due to a ceiling effect, my data are non-normally distributed for both groups. At the recommendation of a statistician/professor, to correct for this problem I bootstrapped the t tests in SPSS using 95% bias corrected confidence intervals based on 1,000 replications. The smaller group has a larger variance, so I am interpreting the analyses using Welch's correction ("equal variances not assumed").

The SPSS output for the bootstrapped t tests produces a table with some p values left blank under the Sig. (2-tailed) column...Why is this? And what do I report instead of these missing bootstrapped p values? I just want to be consistent in how I explain which results were significant.

All of the Lower and Upper BCa 95% confidence intervals ARE listed in the table, but I'm not sure how to succinctly communicate significance using only the confidence intervals (Though I know that if the interval does not include 0 then it can be concluded that the difference between the 2 groups is significant).

• "Due to a ceiling effect, my data are non-normally distributed for both groups." Non-normality is not the issue here. It's the ceiling that you described. With or without bootstrap, those observations are interpreted as achieving the upper limit of detection rather than exceeding it to an unknown extent. Bootstrapping is not a solution to this problem that I'm aware of. If normality would be a reasonable parametric assumption otherwise, the EM algorithm is the correct approach. Commented May 28, 2014 at 19:46

2 Answers

It is possible, even if very rare, a resampled sample to be composed by all same values; so the variance would be zero and the t statistics infinity

• This is in general an answer for any bootstrap test (or multiple imputation test) in SPSS. If one of the subsets did not converge it will not report the pooled analysis. I think from the OP's question they have dichotomous data, so with only 30 some observations this seems like a real possibility. I'm not sure what to say in response that SPSS reports all of the confidence intervals - maybe it reports the interval with zero length in the case of no variance. Commented Jun 29, 2014 at 13:28

I have had the same issue (SPSS v23) with missing p-values when running multiple t-tests. A workaround was to only test the variables with missing values one at a time. I find this odd as the analysis was set to 'analysis-by-analysis' so should have been pairwise. Sample size and mean difference was the same in the output of both analyses but Bootstrap numbers (Bias, SE, Sig, 95%CI) were different or in the case of the p-value missing when the test was run with multiple variables.

• That seems to me to be (a) a comment not an answer (b) not really about the same problem anyway. Commented Aug 16, 2016 at 14:37
• This seems like potentially an answer to me. I think it could stay open. Commented Aug 16, 2016 at 15:50
• Since this contains a workaround, I think it may constitute an answer to the question, as @gung suggests. Commented Aug 16, 2016 at 19:59