Given reference samples $R_i$, I have two dichotomous responses $X_i$ and $Y_i$ I wish to compare. Each dichotomous response (relevant vs. irrelevant to the reference $R_i$) is generated by an algorithm based on $R_i$. I can also ask subjects to rank paired samples by relevance ($X_i ≷ Y_i$). I want know if algorithm $y$ is significantly better than $x$:

$$\begin{align}H_0: r_x &= r_y \\ H_1: r_y &> r_x \end{align}$$

What is the best test to apply? What if there are than two treatments?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.