1
$\begingroup$

I have one interval variable on a 1-25 point interval scale, i.e. placement test scores where 25 is the highest competence score (data not normally distributed), and six different ordinal variables on a 1-7 likert scale (where 7 is the highest score) in my data set. There are 12 participants (12 interval test scores) and 6 raters (each scored 6 different ordinal variables per participant).

Based on what I've read, I've decided to run the Spearman's correlation between the interval and each ordinal variable (i.e. six different correlation tests between the interval variable and each ordinal variable) in SPSS but would it be one-tailed? Or is there any other test I should run to test the level of agreement between the raters in my study?

Thank you for your time and consideration in reading this question!

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ What hypothesis are you testing? That information is necessary for determining whether a one-tailed or two-tailed test should be run. (The data cannot tell you which one would be appropriate.) $\endgroup$
    – whuber ♦
    Commented Jul 7, 2014 at 19:08
  • $\begingroup$ Thank you for your comment. My hypothesis is that the relationship between the two scales is unidirectional in that the higher the score on the 0-25 interval scale, the higher the score on the 1-7 ordinal Likert scale. Would that be a one-tailed Spearman's correlation test? $\endgroup$
    – sb415
    Commented Jul 8, 2014 at 6:03
  • $\begingroup$ You can never prove a scientific hypothesis, so the way to proceed is to use your data to disprove the negation. That is, you would hope to demonstrate that the relationship is neither zero nor negative nor curvilinear. That's the null hypothesis. The alternative, which you will accept provided the null is rejected (and no other evidence emerges casting doubt on the statistical model), is what you hope to demonstrate. The curvilinearity is ruled out by assumption (which you should check). The rest can be tested using the Spearman coefficient; so yes, it's one-tailed. $\endgroup$
    – whuber ♦
    Commented Jul 8, 2014 at 14:09

1 Answer 1

Reset to default
2
$\begingroup$

It is not completely clear what you end goal is, are you interested in the agreement between raters? or in using the ratings to predict the test score? or the test score to predict the rater score? or something else?

There are more options than just correlation for looking at inter-rater agreement, including values that can be calculated on multiple raters, see this Wikipedia article for some starting reading.

When choosing between a one-tailed or two-tailed test, the decision is based on the science and the question, not the data. If you a priori believe that if there is a relationship then it will only be in one direction (or that a relationship in the other direction will have the same results as no relationship) then a one-tailed test is appropriate. If a relationship in either direction is of interest then you should do a two-tailed test, or if you will create a confidence interval and don't want it going to infinity in one direction, then you should do a two tailed test. But note that not all statistics needs to be specifically a test.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for your answer. Sorry, I kind of accidentally blended two questions there-- first, I wanted to check whether a higher interval proficiency test score directly corresponded to a higher ordinal Likert score for global competence; so, judging by your answer, it looks like I should run the one-tailed Spearman's correlation test. Another goal is to check the inter-rater agreement between the six raters for each of the six ordinal variables scored for each of the 24 score sheets passed to each rater (6 ordinal variables per sheet, 24 sheets per rater, 6 raters in this study 6 X 24 X 6). $\endgroup$
    – sb415
    Commented Jul 8, 2014 at 6:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.