0
$\begingroup$

If I were to issue a questionnaire that asks people to provide a particular, continuous numerical score to something (between 0-100) how would I statistically test whether the those scores were within x range of the scores that the same group later provided for something else?

Would using confidence intervals be sufficient?

For example, imagine that I have data at a 95% confidence level in which the scoring bounds for thing 1 are 45 to 65 and for thing 2 they are 71 to 86.

If I was testing for whether thing 1 would be within 5 points of thing 2 would this be sufficient to say it's statistically implausible?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ I'm not sure that confidence intervals are the right tool for this. For clarification: are you primarily interested in comparing mean scores between thing 1 and thing 2, or do you have a different primary goal and are looking at this as well? $\endgroup$
    – Upper_Case
    Sep 9, 2016 at 15:48
  • $\begingroup$ @Upper_Case Thanks for your comment. I'm interesting in comparing mean scores. $\endgroup$
    – user5508297
    Sep 9, 2016 at 15:49

1 Answer 1

Reset to default
1
$\begingroup$

I don't think that confidence intervals are what you want here. What I would do is set up the problem as follows:

  1. Collect the response data for thing 1 and thing 2 (which you've done).

  2. Set up my hypothesis test as follows:

    Null hypothesis: Mean(thing 1) - Mean(thing 2) < 5

    Alternative hypothesis: Mean(thing 1) - Mean(thing 2) >= 5

  3. Execute your chosen test (I'm assuming a two-sample t-test is what you would use)

  4. Assess and interpret your test results at your chosen confidence level (95%, in your case)

If the p-value from your t-test is greater than 0.05 you can say that, based on your sample and at a 95% confidence level, there is not sufficient evidence to conclude that the means of thing 1 and thing 2 differ by five or more.

This is not the same as saying that it is implausible that they do differ by that much, but rather that the evidence from your sample does not support the conclusion that they differ by that much.

Improve this answer
$\endgroup$
7
  • $\begingroup$ how would I express the alternative hypothesis in a statistical analysis tool like Prism 7? Normally I just add the two columns of data and then run the test but I'm not sure how to set it up to test for a difference $\endgroup$
    – user5508297
    Sep 11, 2016 at 23:53
  • $\begingroup$ Should I be looking at the mean of differences? $\endgroup$
    – user5508297
    Sep 12, 2016 at 0:07
  • $\begingroup$ I'm not familiar with Prism 7, but I would be surprised if the documentation for it didn't list a way to perform a two-sample t-test with a difference in means. $\endgroup$
    – Upper_Case
    Sep 12, 2016 at 1:41
  • $\begingroup$ It would be difference in means for a 2 sample t-test but the mean of the differences that I should be looking at in a paired test, right? $\endgroup$
    – user5508297
    Sep 12, 2016 at 1:46
  • $\begingroup$ That sounds like a correct description of a paired t-test. But to be clear: as outlined in your question, this is not a scenario where a paired test is appropriate. $\endgroup$
    – Upper_Case
    Sep 12, 2016 at 13:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.