# How do I correct for gender bias for Bayesian credible intervals?

Forgive me if this question is answered elsewhere - it may be all I need to be told is the correct terminology.

I have a survey in which I ask a number of Yes/No questions (actually Yes/No/Don't know - but for simplicity I want to assume Yes/No for now).

Say for one question I have 583 say yes and 237 say no. Suppose I had really no idea what the proportion of yes and no would be in the population as a whole (this was reasonably accurate), so that my prior belief can be represented by a uniform distribution over the proportion who would say yes in the population, then, if I have understood Bayesian inference properly, I obtain a Beta posterior distribution. I can compute credible intervals from that (using scipy.stats.beta.interval(0.95, 583, 237) for instance).

(For Yes/No/Don't know I can use a Dirichlet distribution).

But in fact my survey includes men and women. Consistently women are less likely to say yes than men. In my sample the yes v no numbers are: 172 v 118 (for women) and 411 v 119 (for men). As you can see more men answered the survey than women. I'd like to correct for that but I don't know how.

Eg, I could fractionally weight the men's answers by the ratio of men/women. The problem is that I don't know how to compute credible intervals for the weighted numbers that would result.

When I try to approach this from first principles, I get stuck working out what a good prior is. If P(M) is my prior belief that the proportion of men who would say yes in the population is M (and P(F) has the corresponding meaning) with m and f representing the sample values then I am computing P(M, F | m, f) via Bayes' theorem and I need a prior for P(M, F).

Although I have no real knowledge about Yes/No proportions for men or women individually I am quite sure they are correlated in some way. Are there well know priors that can incorporate that weak prior knowledge? Or can I avoid working out the whole problem and just do some kind of weighting?

Telling me where to look would be very useful.

• Woman are less likely to say yes than man? The figures go: 226 vs. 52 = 81% yes for women, 365 vs. 111 = 77% yes for men. So it seems you have more 'yes'es from men, because you have more men. And a 4% difference might not even be that much relevant. – Rolazaro Azeveires May 3 '14 at 18:59
• Ooops! That was obviously not a very interesting row (I have several sets of data and one is much less interesting than the others - I keep looking at it and getting confused). I've put in another set of numbers I've collected. It shouldn't make much difference to the approach (I'd have thought). – Francis Davey May 3 '14 at 22:22
• OK. Now its two of us hopping for a reply. – Rolazaro Azeveires May 6 '14 at 23:25

This shouldn't be too difficult to deal with from first principles, simply by treating the parameters for males and females separately and then combining them to give a corresponding estimate of the overall probability of a "yes" (which is a linear function of these parameters). For simplicity, I would suggest you don't worry about having correlation in your prior; you have a substantial amount of data so the value of complicating your model for a minor variant in prior is minimal. As a simple case, you could consider using the following conjugate Bayesian model:

\begin{aligned} X_M &\sim \text{Bin}(n_M, \theta_M), \\[4pt] X_F &\sim \text{Bin}(n_F, \theta_F), \\[4pt] \theta_M &\sim \text{Beta}(\alpha, \beta), \\[4pt] \theta_F &\sim \text{Beta}(\alpha, \beta), \end{aligned}

where the values $\alpha$ and $\beta$ are specified by your prior belief, and $n_M$ and $n_F$ are fixed by the sampling design. This yields independent posteriors:

\begin{aligned} \theta_M | x_M, x_F &\sim \text{Beta}(\alpha + x_M, \beta + n_M - x_M), \\[4pt] \theta_F | x_M, x_F &\sim \text{Beta}(\alpha + x_F, \beta + n_F - x_F). \end{aligned}

For a large population with a specified proportion $\phi$ of males, the overall proportion of "yes" people is given by the parameter $\theta \equiv \phi \cdot \theta_M + (1-\phi) \cdot \theta_F$. The posterior distribution for this parameter is complicated by the fact that it involves a convolution of beta distributions, which does not have a closed-form solution. Nevertheless, it is simple to simulate values of the distribution using the beta random variables.

#Set your prior parameters
alpha <- 1;
beta  <- 1;

nm <- 530;
nf <- 290;

#Input the observed data
xm <- 411;
xf <- 172;

#Simulate posterior parameter
#S is the number of posterior simulations you want
#phi is the proportion of males in the population
S       <- 10^6;
phi     <- 0.5;
theta_m <- rbeta(S, shape1 = alpha + xm, shape2 = beta + nm - xm);
theta_f <- rbeta(S, shape1 = alpha + xf, shape2 = beta + nf - xf);
theta   <- phi*theta_m + (1-phi)*theta_f;


This R code generates a vector of independent simulated values of $\theta$ from the posterior distribution. From there you can estimate the posterior distribution via kernel density methods and obtain the posterior mean/mode or credibility intervals.