Stack Exchange network consists of 183 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.
In retrospect introducing that $\phi$ was very confusing. I have modified it; it simply should be $\theta$. Please let me know if anything else about my question makes it unclear. I can try to rewrite it if necessary.
@MikeRepass what I am saying is: can't you have one Kendall Tau for Q1, Q2, etc? That is, put all the queries together, have your algorithms rank them and then calculate one Kendall Tau.
can your algorithms T and I just rank everything and you compute the Kendall Tau coefficient using the rankings of all queries? or am I misunderstanding?
I am not sure I fully understand what you are saying, but I'll try to reply. The assumption is that the curriculum hasn't changed, namely, perfect students need as many semesters in 1950 and 2000. The reason why there is a change is because the distribution of students changes and that is what I want to test.
Yes, but in practice one doesn't have access to $P$. The authors do indicate that probability is given by: $q_i(\tau,\theta)=\prod_{h=0}^H\pi_i\prod_{h=0}^HP_i$. Do you know how to estimate that value in practice?
It isn't different, it is what you described. When replacement is allowed, then the first batch might be $\{x_i\}_{i=0}^{31}$ and the second batch could contain elements from the first batch, for example, $\{x_i\}_{i=27}^{58}$.
