# Estimating standard deviation in Poisson regression

I'm interested in an estimator of the standard deviation in a Poisson regression. So the variance is

$$Var(y)=\phi\cdot V(\mu)$$

where $\phi=1$ and $V(\mu)=\mu$. So the variance should be $Var(y)=V(\mu)=\mu$. (I'm just interested in how the variance should be, so if overdispersion occurs ($\widehat{\phi}\neq 1$), I don't care about it). Thus an estimator of the variance should be

$$\widehat{Var}(y)=V(\widehat{\mu})=\widehat{\mu}$$

and an estimator of the standard deviation should be

$$\sqrt{\widehat{Var}(y)}=\sqrt{V(\widehat{\mu})}=\sqrt{\widehat{\mu}}.$$

Is this correct? I haven't found a discussion about standard deviation in the context with Poisson regression yet, that's why I'm asking.

Example:

So here is an easy example (which makes no sense btw) of what I'm talking about.

data1 <- function(x) {x^(2)}
numberofdrugs <- data1(1:84)
data2 <- function(x) {x}
healthvalue <- data2(1:84)
plot(healthvalue, numberofdrugs)
test <- glm(numberofdrugs ~ healthvalue, family=poisson)
summary(test) #beta0=5.5 beta1=0.042
mu <- function(x) {exp(5.5+0.042*x)}
plot(healthvalue, numberofdrugs)
# the purple curve is the estimator for mu and it's also
# the estimator of the variance,but if I'd like to plot
# the (not constant) standard deviation I just take the
# square root of the variance. So it is var(y)=mu=exp(Xb)
# and thus the standard deviation is sqrt(exp(Xb))
sd <- function(x) {sqrt(exp(5.5+0.042*x))}
curve(sd, col="green", lwd=2)


Is the the green curve the correct estimator of the standard deviation in a Poisson regression? It should be, no?

• Standard deviation of what? In Poisson regression, $\mu$ varies with the independent variables. Thus, unless the slope estimate is exactly zero, there is no fixed value of $\mu$. Are you perhaps looking for some analog of the standard deviation of the regression residuals? – whuber Jul 9 '11 at 11:32
• @whuber's point is valid. Please clarify. I am guessing you are trying to find the $\mu$? From a Bayesian perspective you can obtain a posterior distribution for $\mu | \mathbf{y}$. – suncoolsu Jul 9 '11 at 11:54
• you have 3 similar posts. Could you please give us a better picture of what your data are and what are you trying to answer. Standard deviation doesn't make a lot of sense. We want to know what's the S.D. for. – suncoolsu Jul 9 '11 at 11:57
• The follow-up Q was so similar that I've merged it with this. – user88 Jul 9 '11 at 16:10
• @MarkDollar I've merged your two questions as the second one was merely an illustration for the present question. It's better to keep everything in the same thread. – chl Jul 10 '11 at 17:18

Poisson regression finds a value $\hat{\beta}$ maximizing the likelihood of the data. For any value of $x$, you would then suppose $Y$ has a Poisson($\exp(x \hat{\beta})$) distribution. The standard deviation of that distribution equals $\exp(x \hat{\beta}/2)$. This appears to be what you mean by $\sqrt{\widehat{\mu}}$.

There are, of course, other ways to estimate the standard deviation of $Y|x$. However, staying within the context of Poisson regression, $\exp(x \hat{\beta}/2)$ is the ML estimator of SD($Y|x$) for the simple reason that the ML estimator of a function of the parameters is the same function of the ML estimator of those parameters. The function in this case is the one sending $\hat{\beta}$ to $\exp(x \hat{\beta}/2)$ (for any fixed value of $x$). This theorem will appear in any full account of maximum likelihood estimation. Its proof is straightforward. Conceptually, the function is a way to re-express the parameters, but re-expressing them doesn't change the fact that they maximize (or fail to maximize, depending on their values) the likelihood.

• That sounds good. Thus my guess was right. – MarkDollar Jul 14 '11 at 10:45

You are thinking too much in terms of "normally distributed" here. For a normal distribution, you have two parameters then mean $\mu$ and the variance, $\sigma^2$. So you require two pieces of information to characterize the probability distribution for the normal case.

However, in the Poisson distributed case, there is only one parameter, and that is the rate $\lambda$ (I relabeled to avoid confusion with normal). This characterizes the Poisson distribution, and so there is no need to refer to other quantities.

This is why probably why don't hear standard deviation "estimation" mentioned in Poisson regression. Asking for a standard deviation estimator for a Poisson random variable is analogous to asking for a kurtosis estimator for a normally distributed random variable. You can get one, but why bother? By estimating the rate parameter $\lambda$, you have all the information you need.

• It is very interesting then that there exist several different kurtosis estimators for normal variables (all based on the same estimates of $\mu$ and $\sigma$)! Do you suppose this could be what is motivating the original question? :-) – whuber Jul 13 '11 at 15:45
• That there exists multiple estimators does not provide an answer to why one should try to "estimate" the kurtosis, given you have already estimated the mean and variance. What value does it add, given we aren't questioning over-dispersion (as the OP stated)? – probabilityislogic Jul 13 '11 at 16:22
• The point to keep in mind is that the O.P. is concerned about estimating a standard deviation and that there is no a priori reason to suppose the "obvious" estimator based on an estimated mean is best or even good. Even though one parameter may characterize the Poisson and two may characterize the Normal distribution, that does not imply that estimators of those parameters solve all estimation problems involving those distributions! (If it did, we could dispense with the entire theory of estimation...) – whuber Jul 13 '11 at 17:36
• @probabilityislogic, I like the kurtosis analogy. (If you stipulate that you assume your sample is normal, there's nothing to estimate — the excess kurtosis is 0 by definition). @whuber, I'm intrigued by your answer: I was not aware of any estimators for kurtosis from normal RVs. (I can't fathom why one would want such a thing, except perhaps for a test of non-normality). Can you provide more intuition, or perhaps an example? – jpillow Jul 13 '11 at 17:54
• @whuber But at the moment, we have no reason to suppose that it is bad either. And I am not referring to the absence of any indication of error in the estimate that is given to $\lambda$. But the error in the estimate for $\lambda$ is not the same thing as the standard deviation. From my perspective, if $\lambda$ was known with certainty, what is the use of a std dev estimator? For normal, if mean is known, then standard deviation is still worth estimating. But, if mean and std dev of a normal are known, what is the use of a kurtosis estimator? – probabilityislogic Jul 13 '11 at 18:01