describing binomial data in likelihood models

Question

Binomial data can be described in various ways. Suppose we flip a fair coin twice and get one head (success). One method to calculate its negative log-likelihood is

-dbinom(1, 2, 0.5, log=TRUE)

Another approach is

-sum(dbinom(c(1, 0), 1, 0.5, log=TRUE))

Consequently, the results differ, influencing model selection. If each Bernoulli trial represents an independent replication, N replications can be divided in numerous ways (for instance, when N=5, as 1+1+1+1+1 or 2+3, or 1+4, etc.). Due to the differing likelihoods, these divisions are not interchangeable. However, is one method of describing the data superior to others?

Related code in R

> x <- c(2,4)
> N <- c(10,10)
> x0 <- rep(c(1,0), c(sum(x) ,sum(N)-sum(x)))
> 
> AIC(glm(cbind(x,N-x) ~ 1, family=binomial))
[1] 8.127032
> AIC(glm(x/N ~ 1, weights=N, family=binomial))
[1] 8.127032
> AIC(glm(x0 ~ 1, family=binomial))
[1] 26.43457
> AIC(glm(cbind(x0,1-x0) ~ 1, family=binomial))
[1] 26.43457

Answer 1

The two approaches model different situations. In the first case, we do not observe the order of the values $0$ and $1$. All we know are the totals: how many zeroes and ones we observed, and that's what we are modeling—the totals. In the second case, we observe the values of the individual observations, and that's what we're modeling.

To see why you get different results, consider the simple case of two observations, as in your first example. The possibilities - all equally probable if the underlying probability of a $1$ is $0.5$ - are:

$$\begin{eqnarray} (0,0): \mathrm{sum = 0} \\ (0,1): \mathrm{sum = 1} \\ (1,0): \mathrm{sum = 1} \\ (1,1): \mathrm{sum = 2} \end{eqnarray}$$

When we model the sum, we see a $50\%$ probability of observing a one; when we model the sequence, we see a $25\%$ probability of observing the sequence $(1,0)$. To repeat: this is because we are modeling different things.

Which model is better? It depends upon the data-generating process and your modeling objectives. In cases where the Binomial distribution is truly applicable - iid Bernoulli variates - the sequence is irrelevant, so use the Binomial model. In situations where, for example, the probability changes from observation to observation as a function of exogenous variables, then the Binomial distribution isn't applicable, and the "sequence of Bernoulli distributions" model is appropriate.