I asked this question on stackoverflow first since it is more of an implementation question, but I got no answers yet, therefore I'm asking it here again.
I have a dataset which is annotated by 2 different sources. This means that for every sample, 2 different sources give a label. I want to see how similar the two sets of labels are. Therefore I thought of using the Spearman correlation and the test for homogeneity with chi squared.
I made this question because it is the first time I'm doing this and also in Python. So I'm not sure I'm doing them right, and would like some feedback.
For example I have my 2 sets:
set1 = [1,5,7,3,2,4,...] set1 = [1,3,7,2,2,1,...] rho, pval = scipy.stats.spearmanr(set1,set2)
The results are:
rho = 0.456498145813 p value = 0.0
What do these results mean? (I think there is very little correlation, though why do I have a $p$ value of 0?)
And for the test for homogeneity I calculate the frequencies and use the chi squared test:
chisq , p = scipy.stats.chisquare(np.array(distributionSet1),np.array(distributionSet2),6)
The results here are:
chisq = 47611.7764023 p = nan
How can the $p$ value be
nan? And what does such a high chi squared value mean?
For the degrees of freedom, I calculated 6 (I have 2 sets, so 2 populations, and 7 categories, so $(2-1)\times(7-1)=6$).