0
$\begingroup$

A study aims to determine if a new tutoring program (method 1) is better than standard preparatory program (method 2) to prepare students for the SAT. It measures the proportion of students who score in the top quarter of test takers. Results: 53% of sampled students using the standard program (method 2) scored in the top quarter of test takers, while 45% of sampled students using the new program (method 1) scored in the top quarter of test takers. A hypothesis test will be performed using a 5% significance level.

  1. Will the test statistic be positive or negative?
  2. True or false: At a 5% significance level, the results are statistically significant.

I put negative for #1 and false for #2, but I got both of them wrong. Can someone explain how I solve this type of question in the future and, most important, avoid choosing incorrectly?

Improve this question
$\endgroup$
4
  • $\begingroup$ If this relates to some course, could you please tag as homework? Please also tell us your reasoning/work on your answers, so we can more readily explain what you don't know, rather than spending time telling you things you do know. $\endgroup$
    – Glen_b
    Commented Mar 1, 2013 at 4:07
  • $\begingroup$ 1. The test statistics is likely to be Pearson's chi-squared, which is always positive. 2. Not sure what the question means. :( $\endgroup$
    – Penguin_Knight
    Commented Mar 1, 2013 at 4:37
  • $\begingroup$ How about 2.? It seems like there isn't enough information to determine this, but surely there has to be. $\endgroup$
    – Bob John
    Commented Mar 1, 2013 at 4:39
  • $\begingroup$ @BobJohn Try reading this thread $\endgroup$
    – Penguin_Knight
    Commented Mar 1, 2013 at 4:43

1 Answer 1

Reset to default
1
$\begingroup$

1) It depends on the test statistic! You can sensibly get positive or negative test statistics, but the two I'd most likely use (two sample proportions test of $\pi_2-\pi_1$ OR chi-square) would both be positive.

2) There's not enough information - we don't have sample sizes!

[Well, in fact we sort of can sometimes figure out a difference must be significant even without the sample sizes, just based on the counts that can give us the observed proportions:

i) What's the smallest sample size in which we could observe 45%? Assuming it's a rounded fraction, it looks like 5/11

ii) what's the smallest sample size in which we could observe 53%? That looks like 8/15 is the smallest possible.

So if it's significant at the lowest possible sample size, and the next few 45%-vs-53% sample-sizes up (to allow for the fact that the actual difference might get a little smaller and more than undo the gain in smaller standard errors), it will be significant at larger samples. But in this case it turns out it's not significant for 5/11 vs 8/15:

> prop.test(c(5,8),c(11,15))

2-sample test for equality of proportions with
continuity correction

data:  c(5, 8) out of c(11, 15) 
X-squared = 0, df = 1, p-value = 1
alternative hypothesis: two.sided 
95 percent confidence interval:
 -0.5452923  0.3877165 
sample estimates:
   prop 1    prop 2 
0.4545455 0.5333333

So that's no help - since it's not significant at the smallest sample sizes, we still can't tell for sure. I assume you have sample size information elsewhere that you didn't notice.]

Improve this answer
$\endgroup$
3
  • $\begingroup$ +1 for the glorious and futile attempt at deducing part 2 by considering the fractions involved! $\endgroup$
    – Silverfish
    Commented Mar 12, 2014 at 13:04
  • 1
    $\begingroup$ @Silverfish It's surprising how often the "only have percentages without the sample size" issue comes up (e.g. trying to test a hypothesis based on some information in a report that only gave the percentages in a table). Once in a while (either due to a large difference in percentage or a percentage with a large minimum denominator) you can get significance. Even one more significant figure dramatically improves the chance of getting a significant difference. I worked a table out for whole percents one time, and there were only a few pairs for which you could get significance at the 5% level. $\endgroup$
    – Glen_b
    Commented Mar 12, 2014 at 21:40
  • 1
    $\begingroup$ ... If I remember right, I think I previously had a one sample case work out here (i.e. that the smallest denominator that would yield the observed percentage was still significantly different from the null hypothesis proportion). Most people say "sorry, you can't do anything and that's that"... but there is at least this one thing that can be tried, and once in a while works out, even with percentages rounded to whole numbers. $\endgroup$
    – Glen_b
    Commented Mar 12, 2014 at 21:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.