I need help in my choice of a prior for a bayesian model.
I have data from a set of participants responding to a set of yes/no questions. Answers are correct or incorrect. I suspect some questions are harder than others. I also suspect that some participants are smarter than others. Some of the variance in question difficulty might be explained by question category which I know. Some of the variance in participant smartness might be explained by participant age which I also know.
In a frequentist framework I use a hierarchical logistic generalised linear regression model which predicts response to questions (correct / incorrect) from question category and participant age as population-level predictors with a grouping factor for participants and for questions.
(Age effects might vary across questions and category effects might vary across participants but let's not go there now.)
I'm uncomfortable with the frequentist approach because my participant pool and question pool are both finite. A p value of 0.05 (or whatever) is not trivial to interpret.
As a result I switched to a Bayesian GLMM. This is now easy to do, even for people like me. It has the same description and uses a bernoulli error distribution (recommended by Stan for binary data) and a logit link function. I'm unsure about the choice of a prior for the intercept and for the beta. I don't have great convictions on the prior and want to keep it weakly informative.
- I first used a noninformative prior, because that's closest to the frequentist mechanics. I can generate the posteriors just fine (based on the WAMBS checklist). But this is generally advised against and, to be honest, I don't actually think that the responses and the effects have infinite variance and can be just about anywhere.
- I could assume a normal distribution as my prior, or perhaps a student distribution to be a bit stricter. This will be wilfully ignorant of the process that I think is generating my data but it probably won't hurt much.
- I could use a binomial distribution?
- I could use a beta distribution? That is apparently the conjugate prior for a Bernoulli likelihood function but I'm not sure how much I should worry about that. Also I'm not sure how to keep such a prior weakly informative but I can look it up.