The data I want to analyze consist of set of votes similar to the voting system here on stackexchange. Votes are binary, i.e, items can receive up- or down-votes. The data have been gathered in an A/B controlled experiment.
I want to compare the control group to the treatment group according to some gold standard. That is, for some items I know what the "correct" vote (according to the gold standard) should have been. However, the number of votes per item varies and can be different between control and treatment group. My hypothesis is, that the treatment reduces the voting "error". But I am not sure, which statistical test to use.