Probability distribution for proportions

Question

I'm trying to build a Monte Carlo simulation of a production process, and one of my random variables is team productivity. Team productivity is defined as the ratio of the finished requests over the total number of requests for a time period. Saying that, team productivity is always a positive real number between 0 and 1.

When I plot historic data of productivity it suggest a Normal Distribution, however if I generate a normal random variate I will occasionally obtain negatives or even productivity values over one. Can I use a Normal distribution -and implement some mechanism for invalid values- or there are more suitable distributions for proportion data?

Answer 1

A very basic model for this could look something like this.

Let $N_i \sim f(n \mid \theta_i)$ be the total number of projects of team $i$. A team finishes a project with probability $p_i$. Assuming independence among projects (which in practice won't be true) we have that the number of finished projects $K_i$ conditioned on $N_i$ is a binomial $K_i \mid N_i \sim Binom(N_i, p_i)$.

Now to allow heterogeneity among teams put a prior distribution on $p_i$; a common distribution for a probability is the $Beta(\mu, \phi)$ distribution with mean $\mu$ and precision (inverse proportional to variance) $\phi$. For large $\phi$ it looks a lot like the bell shape of a Normal distribution; for $\phi = 1$ and $\mu = 0.5$ it is a uniform distribution on $[0, 1]$; for small $\phi$ it becomes U-shaped. The parametrization with mean and precision by Ferrari and Cribari-Neto (2004) is different to the usual parametrization of the Beta distribution with $\alpha$ and $\beta$, but $\mu$ and $\phi$ is easier to interpret and easier to model in a regression (unless you are Bayesian ;) ). The R package betareg implements estimators for it.

So overall a basic model for your process is $$ N_i \sim Poisson(\lambda_i) \\ K_i \mid N_i \sim Binom(N_i, p_i) \\ p_i \sim Beta(\mu, \phi) $$

The nice part of the Beta model as Ferrari and Cribari-Neto propose it is that you can easily model the average probability of success of the company as a function of explanatory variables $X$ using a link function (usually logit) to do $\mu = logit^{-1}(X \beta)$. See this post for an example (with illustration plot of the probability as a function of $x$).

As an extension to this consider the Poisson-binomial distribution, where the success of each project for a team is not equal.

Update: Simulation code looks something like this (here I model the probability of success as a function of team size):

nTeams <- 100
avg.team.size <- 10
team.size <- rpois(nTeams, avg.team.size)
dev.from.avg.size <- abs(team.size - avg.team.size)^2
prob.mu <- exp(-.01 - dev.from.avg.size)
prob.phi <- dev.from.avg.size + 4

prob.alpha <- prob.mu * prob.phi
prob.beta <- (1 - prob.mu) * prob.phi

pp <- rbeta(n = nTeams, shape1 = prob.alpha, shape2 = prob.beta)

NN <- rpois(nTeams, lambda = 10)
KK <- rbinom(length(NN), size = NN, prob = pp)

One thing to take care of is that the original paper works only for $p \in (0, 1)$, i.e., the open interval. Binomial regression does not have this issue, and if you have access to $N$ and $K$ observations then I suggest to follow Glen_b advice and do binomial. However, there are extension to beta regression for $p \sim [0, 1]$.

An interesting question to ask is the following: how many projects could a team have completed had they not had $p < 1$ success rate? This is related to imputation models and the Beta-Binoial Negative-Binomial model (see Hofler and Scrogin and Goerg et al.).

Answer 2

Count proportions are often modelled as (conditionally) binomial -- or rather as scaled binomial; if $X\sim\text{Bin}(n,p)$ is the count of finished requests, then $X/n$ is the sample proportion. More sophisticated models would look at the fact that the requests are not homogeneous (among other sources of potential heterogeneity) so you might end up with say mixtures of binomials (where the mixing is due to the causes of heterogenity. If you aren't trying to model the sources of heterogeneity, you might consider, say a beta-binomial).

You might want to also consider time dependence (e.g. slowing effects of say fatigue or burnout vs speedups from learning effects of doing similar jobs in a row)