What statistical test is appropriate for the following?

I have student ranking by year (1=freshman, etc), along with their responses to questions that seek to find how developed they are in a number of skills (e.g. for team skills, a 3 would indicate excellent team skills, 1 would be not as developed.) I am hypothesizing that a student who has been in school longer will have a higher score for each skill.

It has been suggested that I use chi-squared because of the type of data, but I am not sure if this is right. I am predicting a difference for each year in school, but I am making a directional prediction.

What would you suggest?

  • $\begingroup$ One good option is Jonckheere-Terpstra test. You can learn more about it if you search this site. $\endgroup$ – ttnphns Jul 6 '15 at 22:18

You are basically interested in testing for concordance, which means that when one quantity (year) increases the other (skill) does as well. There are various tests you could use, but one is based on something known as Kendall's $\tau$, which for two random quantities $X$ and $Y$ is defined as $P(\text{$X$ and $Y$ are concordant}) - P(\text{$X$ and $Y$ are discordant})$.

Each of these probabilities can be estimated in a straightforward way from the data and a significance test done to determine if $\tau > 0$. (If the sample size is sufficiently large one can use a normal approximation to the null distribution of $\hat{\tau}$.) Most statistical software packages have methods for performing this test. You can find more information here: https://en.wikipedia.org/wiki/Kendall_rank_correlation_coefficient


You can make student rank to be an ordered factor (hence clarifying that 1, 2 and 3 are categories but with 3>2>1) and use simple linear regression to determine if there is a relation, as in following simple example. The data has been arranged as follows:

  rank answer
1    1      1
2    1      2
3    1      1
4    2      2
5    2      2
6    2      1
7    3      3
8    3      2
9    3      2

The rank is an ordered factor where 1,2 and 3 have a constant increase. The answers are also ordered but for simple regression can be considered as numeric:

 'data.frame':   9 obs. of  2 variables:
 $ rank  : Ord.factor w/ 3 levels "1"<"2"<"3": 1 1 1 2 2 2 3 3 3
 $ answer: int  1 2 1 2 2 1 3 2 2

On performing linear regression with answer as dependent variable and rank as predictor, one can test the relation between the two. The output in R is as follows:

lm(formula = answer ~ rank, data = mydf)

    Min      1Q  Median      3Q     Max 
-0.6667 -0.3333 -0.3333  0.3333  0.6667 

            Estimate Std. Error t value  Pr(>|t|)    
(Intercept)   1.7778     0.1925   9.238 0.0000909 ***
rank.L        0.7071     0.3333   2.121    0.0781 .  
rank.Q        0.1361     0.3333   0.408    0.6973    
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.5774 on 6 degrees of freedom
Multiple R-squared:  0.4375,    Adjusted R-squared:   0.25 
F-statistic: 2.333 on 2 and 6 DF,  p-value: 0.178

This shows that there is trend towards linear (but not quadratic) relation with adjusted R^2 of 0.25 .

  • $\begingroup$ Just to tack on for OP's sake, what you're really testing here is to see whether or not each additional level of year (i.e. the older the student), is significantly associated with a higher response score, with $H_0$ being no association. The Estimate column will give you the increase in response for each level of year, and the $P$ value will give you the significance of that association. You should also look into multiple testing correction if you are going to be doing this once for each response variable, because then you are testing more than 1 hypothesis at once. $\endgroup$ – Chris C Jul 6 '15 at 16:49
  • $\begingroup$ If the OP isn't familiar with R this post may be a bit daunting; some more initial explanation would be useful $\endgroup$ – Glen_b -Reinstate Monica Jul 6 '15 at 17:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.