1
$\begingroup$

spss paired sample t test

  • Paired samples statistics (table 1): pretest score mean= 41.6667, post test score mean= 93.3333, N=3

  • Paired samples correlations (table 2) Correlations= -.277 , Sig.=.821

  • Paired samples Test (Table 3) t=-7.750, df=2, sig(2-tailed)=.016

PRETEST SCORE

A=30 B=50 C=45

POSTTEST SCORE

A=95 B=95 C=90

The significance level in Table 2 is high while that in Table 3 is low. Should I reject the null or reject the alternative hypothesis in this t-test?

$\endgroup$
1
$\begingroup$

That will depend on which null hypothesis you are testing. At a guess, since this is a t-test, you want to see if the mean changed, and that would be table 3 which is significant (that is, the means changed).

However, I'd be a little leery. There was a huge change not only in the mean scores but in the standard deviation - it was much lower in the post-test (see Table 1). Was this expected? Also, table 2 shows that the correlation between pre and post test is quite low -- in fact, it is negative -- which is odd. You would expect a positive correlation between two tests.

Finally N is only 3.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I believe you can deduce much more. First, you can determine from the output whether the correlation differs significantly from zero. Second, you can apply an F test to establish whether this change in SD is compatible with the data and the implicit assumptions of the test. (If I'm reading the numbers correctly--they appear to be around 10.4 and 2.8--then the p-value for this two sided test is about 15%, suggesting that such a change is not at all surprising.) Finally, could you be explicit about what bearing the final remark "N is only 3" has on rejecting or accepting the null? $\endgroup$ – whuber Dec 1 '19 at 20:27
  • $\begingroup$ One more thing: why would anyone expect a "positive correlation"? If the subjects are independently tested, as they are in most cases, wouldn't one expect zero correlation? This issue is germane because it explains the reason for the first test (in Table 2). $\endgroup$ – whuber Dec 1 '19 at 20:29
  • 1
    $\begingroup$ @whuber, this is a paired t-test. So the 3 subjects are the same in both conditions (before & after?). It is common in such cases for the unit (patient) with the lowest pre-score to have the lowest post-score, etc. We don't know what these data are, but it is common for paired data to be positively correlated. $\endgroup$ – gung - Reinstate Monica Dec 1 '19 at 20:37
  • 1
    $\begingroup$ @gung Thank you: I did not understand the test was correlating pre and post results; I thought it was performing some kind of test of independence of the differences. I would hope, though, that the software is not performing a one-sided correlation test. There are plenty of circumstances where one expects negative correlations (people who were worse off at the beginning made greater gains), indicating the default test ought to be two-sided. $\endgroup$ – whuber Dec 1 '19 at 21:24
1
$\begingroup$

The first table tells you that the pre- and post-average test scores are radically different. The hypothesis you test is that they are indeed different. Such scores are so different as not needing statistical testing for confirmation. If needed, you do have that statistical confirmation in the third table.

The second table has no direct relevance to whether the pre- and post-test average scores are different. It just shows a negative correlation between the two; which is a bit strange. I gather maybe the patients that were in the worst shape improved the most.

The one concern I have is that your sample has only 3 individuals. That seems way too low to derive any sort of reliable inference on the effectiveness of the treatment. You can just observe that based on these 3 patients it overall makes a very big difference. But, you probably need to have a sample of at least 10 x that to derive more reliable results.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Re Table 2: you need to interpret its p-value. Re The final comments: they might over-reach a little. Suppose, for instance, these subjects were three randomly-selected US states, the intervention was a test of a multiyear government program to improve school graduation rates, and the average rates increased from 41% to 93%. Do you think people in the remaining states would agree to let you study another much larger sample before rolling out the program nationwide? The caution about a small $N$ is fair, but this example shows you can't declare that $N$ must exceed 30 to be reliable. $\endgroup$ – whuber Dec 1 '19 at 21:36
  • $\begingroup$ Well Table 2, the negative correlation bit is very far from being statistically significant... which is somewhat of a relief given that this negative correlation may be a bit counterintuitive. I stand by that N = 3 is much too small to make any generalized inference. However, in your example one single data point represents the average for an entire State including millions of underlying data points. That's a pretty different situation from the standard paired t test within a standard hypothesis testing framework (clinical trials, etc.). $\endgroup$ – Sympa Dec 3 '19 at 1:22
  • $\begingroup$ "Pretty different" in what sense? There's nothing in this question or in your answer that would distinguish those two situations. That's why I am suggesting that you not be so definite about your recommendation to use a sample size of 30 or larger. $\endgroup$ – whuber Dec 3 '19 at 14:46
  • $\begingroup$ I think in most hypothesis testing framework, using N = 3 to fully test an hypothesis is way too small. And, suggesting 10 x 3 or a sample of ~ 30 seems acceptable (that's not an inordinately large N). Even in your example at the State level, taking cost and political consideration aside using a much larger sample would be preferable from a statistical standpoint. $\endgroup$ – Sympa Dec 3 '19 at 21:56
  • $\begingroup$ "Taking cost and political considerations aside" shows you're not really trying to solve the statistical problem, but only invoking general rules of thumb and feelings. The statistical standpoint emphatically is the standpoint from which costs, constraints, and objectives are seriously considered in framing and solving the problem; anything else is just empty mathematics. $\endgroup$ – whuber Dec 3 '19 at 22:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.