How can variance be non constant for Bernoulli and Poisson

Question

Until now I had learned that the variances of Bernoulli and Poisson random variables are $p(1-p)$ and $λ$ and that for fixed $p$ and $λ$, these variances are constant.
Now, introducing glm, my course says:
- linear model: $\quad Var(Y_i)=σ^2$, where $σ$ is constant
- binomial model: $Var(Y_i)=μ_i(1−μ_i)$
- Poisson model: $Var(Y_i)=μ_i*$
so the variance of the binomial and Poisson are not constant as I believed but depend on $Y_i$, ie, unless I misunderstand the notation, on the single observation.

I am unable to understand how that can happen, ie why we have a different μi for every observation.
(to be complete I am also unable to understand how a single observation/data-point can have a variance, statistic that I have always seen defined for a sample, not for a single observation, though I realize that in some contexts one can identify an observation with a specific variance, see my answer to myself help to understand how residual standard deviation can differ at different points on X)

Can someone help me to understand this or point to eventual mistakes in my understanding?

Answer 1

The point is that, when one says that "The poisson distribution do not have constant variance" (for instance), one is referring to the POisson distribution family. In a statistical analysis, say, we assume that the count we are interested in do have some poisson distribution, with expectation $\lambda$, for some value of $\lambda >0$, but we do not specify the value of $\lambda$, we do estimate $\lambda$ from the data. Since the variance is then also $\lambda$, it is not constant as a function of the mean. The same goes for, mutatis mutandi, the binomial or bernoulli distribution.

Comparing with the situation for normal distributed models, there we have two parameters, $\mu, \sigma^2$, modelling expectation and variance separately. So then, it is a mathematical possibility that the expectation can vary, without the variance varying with it. So, the variance, as a function of the mean, can be a constant. That is mathematically impossible for a poisson model.

So if we, say, want to compare two treatmente (or treatment with a control), if the response variable is poisson distributed, it is mathematically impossible for the treatment to change the mean, without it at the same time changeing the variance. If the response in noramlly distributed, it is mathematically possible for the treatment to change the mean, without it also changing the variance (of course, in a specific experiment, it might well be that the treatment also change the variance).

EDIT

The OP asks in a comment: " "one is referring to the Poisson distribution family" I assume this means, that we have not fixed λ to a specific value but are speaking about all possible infinite λ, and if we fix a value then we have a single variance. Can you confirm? ". Yes, of course, if you specify a fixed value for $\lambda$, say as an example $\lambda=10$, then the variance is constant! The non-constancy comes from not knowing $\lambda$. But, if you specify $\lambda$, then you know completely the distribution of your data (or, at least, you are saying that you know it ...), and then there is no statistics problem left! Then you have a pure probablity problem, not a statistics problem.

Answer 2

Forget GLMs for a second. Imagine you have measured some count variable for individuals across two groups, where the counts in each group are distributed as Poisson.

The first has population mean $\lambda_1$, and the second has population mean $\lambda_2$.

Is the variance the same for the two groups? No. The variance is constant when you keep $\lambda$ the same (stay in the same group) but if you move across to the other group, both the mean and variance change (naturally, since the variable is Poisson-distributed, so the population variance is identical to the mean).

This is actually the simplest case* of Poisson regression.

For a Poisson regression $\text{Var}(Y_i|\mathbf{x}_i) = \lambda_i$ (where $\lambda_i$ is a function of $\mathbf{x}_i\beta$), so the Poisson mean is different at different values of the predictors. Because the Poisson mean changes, of course the variance must change with it (since the variance is equal to the mean).

Considerations are similar for other GLMs- because the variance is related to the parameter that determines the mean, when the mean changes, the variance will change with it.

* well, aside the trivial case of no contributing regressors at all (i.e. constant term only)