3
$\begingroup$

The data I want to analyze consist of set of votes similar to the voting system here on stackexchange. Votes are binary, i.e, items can receive up- or down-votes. The data have been gathered in an A/B controlled experiment.

I want to compare the control group to the treatment group according to some gold standard. That is, for some items I know what the "correct" vote (according to the gold standard) should have been. However, the number of votes per item varies and can be different between control and treatment group. My hypothesis is, that the treatment reduces the voting "error". But I am not sure, which statistical test to use.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

If you know what the correct vote should be, you could start with a simple chi-square or KS test, testing for the number of votes misclassified for each group (treatment and control) as compared to the gold standard.

-Ralph Winters

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.