# How do I correct for gender bias for Bayesian credible intervals?

Forgive me if this question is answered elsewhere - it may be all I need to be told is the correct terminology.

I have a survey in which I ask a number of Yes/No questions (actually Yes/No/Don't know - but for simplicity I want to assume Yes/No for now).

Say for one question I have 583 say yes and 237 say no. Suppose I had really no idea what the proportion of yes and no would be in the population as a whole (this was reasonably accurate), so that my prior belief can be represented by a uniform distribution over the proportion who would say yes in the population, then, if I have understood Bayesian inference properly, I obtain a Beta posterior distribution. I can compute credible intervals from that (using scipy.stats.beta.interval(0.95, 583, 237) for instance).

(For Yes/No/Don't know I can use a Dirichlet distribution).

But in fact my survey includes men and women. Consistently women are less likely to say yes than men. In my sample the yes v no numbers are: 172 v 118 (for women) and 411 v 119 (for men). As you can see more men answered the survey than women. I'd like to correct for that but I don't know how.

Eg, I could fractionally weight the men's answers by the ratio of men/women. The problem is that I don't know how to compute credible intervals for the weighted numbers that would result.

When I try to approach this from first principles, I get stuck working out what a good prior is. If P(M) is my prior belief that the proportion of men who would say yes in the population is M (and P(F) has the corresponding meaning) with m and f representing the sample values then I am computing P(M, F | m, f) via Bayes' theorem and I need a prior for P(M, F).

Although I have no real knowledge about Yes/No proportions for men or women individually I am quite sure they are correlated in some way. Are there well know priors that can incorporate that weak prior knowledge? Or can I avoid working out the whole problem and just do some kind of weighting?

Telling me where to look would be very useful.

• Woman are less likely to say yes than man? The figures go: 226 vs. 52 = 81% yes for women, 365 vs. 111 = 77% yes for men. So it seems you have more 'yes'es from men, because you have more men. And a 4% difference might not even be that much relevant. May 3, 2014 at 18:59
• Ooops! That was obviously not a very interesting row (I have several sets of data and one is much less interesting than the others - I keep looking at it and getting confused). I've put in another set of numbers I've collected. It shouldn't make much difference to the approach (I'd have thought). May 3, 2014 at 22:22
• OK. Now its two of us hopping for a reply. May 6, 2014 at 23:25

This shouldn't be too difficult to deal with from first principles, simply by treating the parameters for males and females separately and then combining them to give a corresponding estimate of the overall probability of a "yes" (which is a linear function of these parameters). For simplicity, I would suggest you don't worry about having correlation in your prior; you have a substantial amount of data so the value of complicating your model for a minor variant in prior is minimal. As a simple case, you could consider using the following conjugate Bayesian model:

\begin{equation} \begin{aligned} X_M &\sim \text{Bin}(n_M, \theta_M), \\[4pt] X_F &\sim \text{Bin}(n_F, \theta_F), \\[4pt] \theta_M &\sim \text{Beta}(\alpha, \beta), \\[4pt] \theta_F &\sim \text{Beta}(\alpha, \beta), \end{aligned} \end{equation}

where the values $\alpha$ and $\beta$ are specified by your prior belief, and $n_M$ and $n_F$ are fixed by the sampling design. This yields independent posteriors:

\begin{equation} \begin{aligned} \theta_M | x_M, x_F &\sim \text{Beta}(\alpha + x_M, \beta + n_M - x_M), \\[4pt] \theta_F | x_M, x_F &\sim \text{Beta}(\alpha + x_F, \beta + n_F - x_F). \end{aligned} \end{equation}

For a large population with a specified proportion $\phi$ of males, the overall proportion of "yes" people is given by the parameter $\theta \equiv \phi \cdot \theta_M + (1-\phi) \cdot \theta_F$. The posterior distribution for this parameter is complicated by the fact that it involves a convolution of beta distributions, which does not have a closed-form solution. Nevertheless, it is simple to simulate values of the distribution using the beta random variables.

#Set your prior parameters
alpha <- 1;
beta  <- 1;

#Set your sample sizes
nm <- 530;
nf <- 290;

#Input the observed data
xm <- 411;
xf <- 172;

#Simulate posterior parameter
#S is the number of posterior simulations you want
#phi is the proportion of males in the population
S       <- 10^6;
phi     <- 0.5;
theta_m <- rbeta(S, shape1 = alpha + xm, shape2 = beta + nm - xm);
theta_f <- rbeta(S, shape1 = alpha + xf, shape2 = beta + nf - xf);
theta   <- phi*theta_m + (1-phi)*theta_f;


This R code generates a vector of independent simulated values of $\theta$ from the posterior distribution. From there you can estimate the posterior distribution via kernel density methods and obtain the posterior mean/mode or credibility intervals.