3
$\begingroup$

Using an independent samples t test, I am comparing the performance of 2 groups on an activity, based on the percentage of items correct (the percentage of total items correct, as well as percentage of items correct for four sub-domains of the activity). The n of one group is 23, and the n of the other is 34.

Due to a ceiling effect, my data are non-normally distributed for both groups. At the recommendation of a statistician/professor, to correct for this problem I bootstrapped the t tests in SPSS using 95% bias corrected confidence intervals based on 1,000 replications. The smaller group has a larger variance, so I am interpreting the analyses using Welch's correction ("equal variances not assumed").

The SPSS output for the bootstrapped t tests produces a table with some p values left blank under the Sig. (2-tailed) column...Why is this? And what do I report instead of these missing bootstrapped p values? I just want to be consistent in how I explain which results were significant.

All of the Lower and Upper BCa 95% confidence intervals ARE listed in the table, but I'm not sure how to succinctly communicate significance using only the confidence intervals (Though I know that if the interval does not include 0 then it can be concluded that the difference between the 2 groups is significant).

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ "Due to a ceiling effect, my data are non-normally distributed for both groups." Non-normality is not the issue here. It's the ceiling that you described. With or without bootstrap, those observations are interpreted as achieving the upper limit of detection rather than exceeding it to an unknown extent. Bootstrapping is not a solution to this problem that I'm aware of. If normality would be a reasonable parametric assumption otherwise, the EM algorithm is the correct approach. $\endgroup$
    – AdamO
    May 28, 2014 at 19:46

2 Answers 2

Reset to default
4
$\begingroup$

It is possible, even if very rare, a resampled sample to be composed by all same values; so the variance would be zero and the t statistics infinity

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ This is in general an answer for any bootstrap test (or multiple imputation test) in SPSS. If one of the subsets did not converge it will not report the pooled analysis. I think from the OP's question they have dichotomous data, so with only 30 some observations this seems like a real possibility. I'm not sure what to say in response that SPSS reports all of the confidence intervals - maybe it reports the interval with zero length in the case of no variance. $\endgroup$
    – Andy W
    Jun 29, 2014 at 13:28
0
$\begingroup$

I have had the same issue (SPSS v23) with missing p-values when running multiple t-tests. A workaround was to only test the variables with missing values one at a time. I find this odd as the analysis was set to 'analysis-by-analysis' so should have been pairwise. Sample size and mean difference was the same in the output of both analyses but Bootstrap numbers (Bias, SE, Sig, 95%CI) were different or in the case of the p-value missing when the test was run with multiple variables.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ That seems to me to be (a) a comment not an answer (b) not really about the same problem anyway. $\endgroup$
    – mdewey
    Aug 16, 2016 at 14:37
  • $\begingroup$ This seems like potentially an answer to me. I think it could stay open. $\endgroup$
    – gung - Reinstate Monica
    Aug 16, 2016 at 15:50
  • $\begingroup$ Since this contains a workaround, I think it may constitute an answer to the question, as @gung suggests. $\endgroup$
    – Silverfish
    Aug 16, 2016 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.