3
$\begingroup$

Until now I had learned that the variances of Bernoulli and Poisson random variables are $p(1-p)$ and $λ$ and that for fixed $p$ and $λ$, these variances are constant.
Now, introducing glm, my course says:
- linear model: $\quad Var(Y_i)=σ^2$, where $σ$ is constant
- binomial model: $Var(Y_i)=μ_i(1−μ_i)$
- Poisson model: $Var(Y_i)=μ_i*$
so the variance of the binomial and Poisson are not constant as I believed but depend on $Y_i$, ie, unless I misunderstand the notation, on the single observation.

I am unable to understand how that can happen, ie why we have a different μi for every observation.
(to be complete I am also unable to understand how a single observation/data-point can have a variance, statistic that I have always seen defined for a sample, not for a single observation, though I realize that in some contexts one can identify an observation with a specific variance, see my answer to myself help to understand how residual standard deviation can differ at different points on X)

Can someone help me to understand this or point to eventual mistakes in my understanding?

$\endgroup$
3
$\begingroup$

The point is that, when one says that "The poisson distribution do not have constant variance" (for instance), one is referring to the POisson distribution family. In a statistical analysis, say, we assume that the count we are interested in do have some poisson distribution, with expectation $\lambda$, for some value of $\lambda >0$, but we do not specify the value of $\lambda$, we do estimate $\lambda$ from the data. Since the variance is then also $\lambda$, it is not constant as a function of the mean. The same goes for, mutatis mutandi, the binomial or bernoulli distribution.

Comparing with the situation for normal distributed models, there we have two parameters, $\mu, \sigma^2$, modelling expectation and variance separately. So then, it is a mathematical possibility that the expectation can vary, without the variance varying with it. So, the variance, as a function of the mean, can be a constant. That is mathematically impossible for a poisson model.

So if we, say, want to compare two treatmente (or treatment with a control), if the response variable is poisson distributed, it is mathematically impossible for the treatment to change the mean, without it at the same time changeing the variance. If the response in noramlly distributed, it is mathematically possible for the treatment to change the mean, without it also changing the variance (of course, in a specific experiment, it might well be that the treatment also change the variance).

EDIT

The OP asks in a comment: " "one is referring to the Poisson distribution family" I assume this means, that we have not fixed λ to a specific value but are speaking about all possible infinite λ, and if we fix a value then we have a single variance. Can you confirm? ". Yes, of course, if you specify a fixed value for $\lambda$, say as an example $\lambda=10$, then the variance is constant! The non-constancy comes from not knowing $\lambda$. But, if you specify $\lambda$, then you know completely the distribution of your data (or, at least, you are saying that you know it ...), and then there is no statistics problem left! Then you have a pure probablity problem, not a statistics problem.

$\endgroup$
  • $\begingroup$ "one is referring to the Poisson distribution family" I assume this means, that we have not fixed λ to a specific value but are speaking about all possible infinite λ, and if we fix a value then we have a single variance. Can you confirm? I feel pedantic but I want to be 100% sure of details and notation, especially considering that my course too often seems not to be so, if I don't solve even minor doubts as soon as possible they will cumulate with the imprecisions I am deterministally sure to find in my course:-) $\endgroup$ – user110848 Feb 10 '17 at 12:49
  • 1
    $\begingroup$ The statement in your edit (quote it below) is huge help, not only for this question but in general. Makes me realize my error of perspective, until now I was "thinking probability" (due to fact that many years ago I studed enjoyed and understood decently probability but not statistics). Thanks again! "But, if you specify λ, then you know completely the distribution of your data (or, at least, you are saying that you know it ...), and then there is no statistics problem left! Then you have a pure probablity problem, not a statistics problem." $\endgroup$ – user110848 Feb 10 '17 at 13:06
2
$\begingroup$

Forget GLMs for a second. Imagine you have measured some count variable for individuals across two groups, where the counts in each group are distributed as Poisson.

The first has population mean $\lambda_1$, and the second has population mean $\lambda_2$.

stripchart illustrating two Poisson samples of size 10 and means 10 and 30 respectively

Is the variance the same for the two groups? No. The variance is constant when you keep $\lambda$ the same (stay in the same group) but if you move across to the other group, both the mean and variance change (naturally, since the variable is Poisson-distributed, so the population variance is identical to the mean).

This is actually the simplest case* of Poisson regression.

For a Poisson regression $\text{Var}(Y_i|\mathbf{x}_i) = \lambda_i$ (where $\lambda_i$ is a function of $\mathbf{x}_i\beta$), so the Poisson mean is different at different values of the predictors. Because the Poisson mean changes, of course the variance must change with it (since the variance is equal to the mean).

Considerations are similar for other GLMs- because the variance is related to the parameter that determines the mean, when the mean changes, the variance will change with it.

* well, aside the trivial case of no contributing regressors at all (i.e. constant term only)

$\endgroup$
  • $\begingroup$ Forgive the more than naive questions: when you write Var(Yi|xi)=λi what does index i denote: one observation (a pair of a y value and acorresponding x value) or one sample (N pairs of x and corresponding y) or i distributions? When you say different xis, do you mean different groups, different xis within the same group, or both cases? $\endgroup$ – user110848 Feb 11 '17 at 6:39
  • 1
    $\begingroup$ Since regression can have continuous predictors, every observation might have a different mean, so in the most general case, it refers to the $i$-th case (the $i$-th observation) which consists of a $y_i$ and a vector of IVs, $\mathbf{x}_i$. $\endgroup$ – Glen_b Feb 11 '17 at 7:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy