I'm having difficulty replicating/deriving a result in GLM's for Binomial data. That is, if $Y \sim Bin(n, \mu)$ and we put the distribution of $Y/n$ into exponential family form (with a dispersion parameter), then the variance function is given by:

$$V(\mu) = \frac{\mu(1-\mu)}{n}$$

This can be found on page six of these lecture slides and on page 116 of Faraway's "Extending the Linear Model with R"

I just don't see that. Here's my approach. Let's start by writing the response variable's pmf in exponential family form. If we write $X = Y/n$ as our response variable, $nX$ is binomial so that

$f(x) = {n \choose nx} \mu^{nx}(1-\mu)^{n-nx}$

$\implies f(x) = exp\bigg[ log({n \choose nx}) + nxlog(\mu) + (n-nx)log(1-\mu) \bigg]$

$\implies f(x) = exp\bigg[ nxlog(\frac{\mu}{1-\mu}) + nlog(1-\mu) + log({n \choose nx})\bigg]$

$\implies f(x)= exp\bigg[ \frac{xlog(\frac{\mu}{1-\mu}) + log(1-\mu)}{1/n} + log({n \choose nx})\bigg]$

$\implies f(x) = exp\bigg[ \frac{x \theta -b(\theta)}{a(\phi)} + c(x,\phi) \bigg]$,

with $\theta = log(\frac{\mu}{1-\mu})$, $b(\theta)=log(1+e^\theta)$, $\phi = 1$, $a(\phi) = 1/n$, and $c(x,\phi) =log({n \choose nx})$ .

Now, to get the variance function, we begin by:

$b'(\theta) = \frac{1}{1+e^\theta} e^\theta $

$b''(\theta) = \frac{e^\theta}{(1+e^\theta)^2}$

We need this in terms of $\mu$ so we plug in $\theta = log(\frac{\mu}{1-\mu})$ and get

$V(\mu) = \frac{\frac{\mu}{1-\mu}}{(\frac{1}{1-\mu})^2} = \mu(1-\mu)$

This is conspicuously missing a $1/n$. What am I missing?


I think I figured it out. It's important to mention that the discussion in Faraway was in the context of IRWLS.

First of all, we can use either the variance of the response or the variance function in our IRWLS implementation. It just represents a scale change: $Var(Y)=V(\mu)a(\phi)$ where $a(\phi)$ is just some constant. So I think Faraway is actually using the Variance of Y.

Second, Faraway was using R, which actually takes the sample proportion, $\bar{Y}=Y/n$, when it fits the model:

If a binomial glm model was specified by giving a two-column response, the weights returned by prior.weights are the total numbers of cases (factored by the supplied case weights) and the component y of the result is the proportion of successes.

So even though $Y\sim Bin(n,\mu)$, the response $\bar{Y}$ has variance $Var(\bar{Y})=\frac{1}{n^2}Var(Y)=\frac{\mu(1-\mu)}{n}$

So until I'm told otherwise, I will assume that his writing $V(\mu)$ above was either an error or done to gloss over something. It's the variance of the response.

| cite | improve this answer | |

There is nothing wrong in your derivation, however, the variance function for exponential family written in your form is


Which is exactly $\frac{\mu(1-\mu)}{n}$.

| cite | improve this answer | |
  • $\begingroup$ I agree that $Var(Y) = a(\phi)b''(\theta)$. But I can find a contradiction to what you wrote above. For the one - dimensional Gaussian, Faraway gives $b(\theta) = 1/2 * \theta^2 \implies b''(\theta) = 1$. Faraway also gives $a(\phi) = \phi = \sigma^2$. Thus we should have $V(\mu) = \sigma^2 * 1$ which contradicts the fact that on page 115 the variance function is stated to be unity; $V(\mu) = 1$. $\endgroup$ – RMurphy Jun 19 '16 at 21:47
  • $\begingroup$ I was working on something else, and I realized the key might be a different parameterization. Letting $\beta = n*\mu$ be the expected number of successes, we can write $\theta = log(\frac{\mu}{1-\mu}) = log(\frac{\beta}{n-\beta})$, which gets us that n in the denom $\endgroup$ – RMurphy Jun 19 '16 at 23:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.