How to analyze within subjects count data? Let's say we have three choices generated by three different algorithms shown to the users repeatedly, and users can choose one among three in several sessions.
Therefore, each user has seen e.g., 12 sessions.
We then want to find out which system generates better choices.
What are the appropriate statistical tests?
I was thinking about the count data and success per trial, but this is a within-subjects assessment. For this evaluation, I captured the choice counts.
 A: Assuming you might have data such as follows (Variable algorithm is encoded because it is a categorical variable):

Since you only recorded the count data, I believe that you would want to use a Generalized Linear Model with a Poisson distribution, perhaps with a random effect of subject.
In R this could be coded as:
library(lem4)

glmer(successes ~ algorithm_2 + algorithm_3 + (1|subject), data = d, family="poisson")

the output in this case would be:
Random effects:
 Groups  Name        Variance Std.Dev.
 subject (Intercept) 0        0       
Number of obs: 9, groups:  subject, 3

Fixed effects:
            Estimate Std. Error z value Pr(>|z|)  
(Intercept)   0.8473     0.3780   2.242    0.025 *
algorithm_2   0.4520     0.4835   0.935    0.350  
algorithm_3   0.9445     0.4454   2.120    0.034 *
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) algr_2
algorithm_2 -0.782       
algorithm_3 -0.849  0.663

If you wanted to estimate the expected number of successes for each algorithm you would use the link function for the Poisson:
# expected number correct for algorithm_1
exp(0.8473)

# expected number correct for algorithm_2
exp(0.8473+0.4520)

# expected number correct for algorithm_3
exp(0.8473+0.9445)

I am not an expert and I would consult a statistician if this is, for example, data for a publication.
If your data are not balanced in the number to repetitions per subject, this paradigm will still work the same.  Your data might look like this:

The model would still be defined exactly as it was above.
Note: that in this data set, there is no variance for the Random intercept of subject, which means it fits exactly the same as a glm without the random effect.
