Statistics for a probability parameter

Question

I am measuring how humans perform a perceptual test. The graph shows the proportion of correct responses as a function of an independent variable x for one person.

The model includes a lapse parameter that corresponds to the asymptotic proportion for x large; for this person the estimated lapse parameter is around 0.94.

Several participants performed the perceptual test and I have an estimation of the lapse parameter for each participant. For many participants the estimated lapse parameter is 1.

How should I perform statistics across participants for the lapses given that they are not normally distributed? Also they are probabilities, but I don't have a size and number of successes for each one. For example,

• how can I calculate confidence intervals for the mean lapse rate across participants?

• how can I perform a t-test or ANOVA type of analysis for the lapses (the participants performed the perceptual test in different conditions)?

Answer 1

Here are my two cents:

1. The way I would go about it is to use a Bayesian multilevel model, to fit psychometric functions for all participants in one go. In this case you could set $\text{Beta} (a,b)$ as a prior for your lapse parameter. Say you estimate $a$ and $b$ via MCMC sampling, then you can easily calculated the expected lapse rate as $\frac{a}{a+b}$, and use any methods (percentile, HPDI) to calculate a confidence interval. In practice, you'd need also an hyperprior for $a$ and $b$, which could be a Gamma distribution (since they both can take only positive values). You can find an example of how to run this approach using Stan here (use left/right arros keys to move through the slides; see also here for a repository containing all the example code).
2. Alternatively, another idea could be to use a Beta regression (see this vignette for example), which is suitable to model dependent variables that are probability themselves. In this case you can just use the lapse rate you have already estimated as a dependent variable.

They both assume that the lapse rates have a Beta distribution, which I think is reasonable. Personally I think (1) is better, since it does take into account the subject-specific standard error in estimating the group-level distribution of lapse rates. This would be particularly helpful if, for example, you have different number of trials per subject. If your dataset is perfectly balanced then (1) and (2) should give similar results.