I used the Friedman test to compare three repeated measures and I use Kendall’s Coefficient of Concordance (W) to get its effect size.
SPSS uses this formula: F (from Friedman test)/n_f (k-1), and I implemented it like and it returns similar value to SPSS:
stat/(k*(m-1))
I also used python, however, using the original Kendall’s Coefficient of Concordance (W) formula, but it yields very different outputs. Why is that and which one is mathematically correct?
Here is my function in python:
def my_friedman_test(q1, q2, q3, expt_ratings):
stat, p = friedmanchisquare(q1, q2, q3)
print('stat=%.3f, p=%.3f' % (stat, p))
if p > 0.05:
print('Friedman Test: Probably the same distribution, the distributions of all samples are equal.')
else:
print('Friedman Test:Probably different distributions, the distributions of one or more samples are not equal.')
# Kendall’s W
if expt_ratings.ndim!=2:
raise 'ratings matrix must be 2-dimensional'
m = expt_ratings.shape[1] #raters
k = expt_ratings.shape[0] # items rated
print(stat/(k*(m-1)))
rating_sums = np.sum(expt_ratings, axis=1)
S = k*np.var(rating_sums)
W= (12*S)/((m**2)*(k**3-k))
print ("raters: ",m," items rated: ", k, "and W= ",'%.8f' % W)
print("---------------------------------------------")
return