# Convergence of poisson distribution

Let $$X\sim \operatorname{Pois}(\lambda)$$ and $$x_1,\ldots,x_n$$ observations following this distribution. I want to derive the analytical solution of the following series: $$\ell(\lambda):=\lim_{x\rightarrow \infty} \frac{1}{n}\sum_{i=1}^n \log P(X=x_i).$$

EDIT: After a few trials, I found a good numerical approximation of the solution: $$\ell(\lambda)=-\frac{1}{2}\log(17.08\cdot \lambda).$$ See the graph below, where the dots represent an approximation of the solution by simulating poisson distributions, while the blue line represents the approximated numerical solution.

x=1:1000
y=sapply(x,function(x) mean(log(dpois(rpois(100000,x),x))))
plot(x,y)
lines(x,-log(sqrt(x*17.08)),col="blue")


• That's right on the mark: the constant is $2\pi e \approx 17.07947.$ See the formula for entropy at en.wikipedia.org/wiki/Poisson_distribution. As a check, follow up with curve(-(log(2*pi*exp(1)*x)/2 - 1/(12*x) - 1/(24*x^2) - 19/(360*x^3)), add=TRUE, col="Red", lwd=2, n=501) as suggested by the formula at that page. BTW, for faster iteration you don't need such a tightly spaced set of $\lambda.$ Use, say, x=exp(seq(0, 20, by=1/2)) and include log="x" as an argument to plot. – whuber Feb 2 at 16:52
• Thanks whuber, I didn't know that what I wanted to calculate was actually the entropy! If you had added an answer instead of commenting my question, I could have selected it as the right answer. – Anthony Hauser Feb 3 at 16:45
• Your use of the word "serie", with no final "s" makes me wonder if you're more accustomed to French than to English. Or was that just a typo? $\qquad$ – Michael Hardy Feb 3 at 19:15

If you have $$n$$ samples of a random variables $$Y$$ (or equivalently, $$n$$ iid random variables) and you compute the sample mean as

$$m_n = \frac{1}{n} \sum_{i=1}^n Y_i$$

then, as $$n$$ grows, (under certain conditions) $$m_n$$ will converge (in some sense) to the expected value of $$Y$$, that is $$m_n \to E[Y]$$. See the law of large numbers, which clarifies those "certain conditions" and in what sense the "convergence" occurs.

In our case, $$Y=\log(p(X))$$ hence

\begin{align} m_n \to E[\log(p(X)]&=\sum_x p_X(x) \log(p_X(x)) \\ &= \sum_{x=0}^\infty e^{-\lambda} \frac{\lambda^x}{x!} (-\lambda + x \log(\lambda) - \log(x!)) \\ &= -\lambda (1 - \log(\lambda)) - e^{-\lambda} \sum_{x=0}^\infty\frac{\lambda^x}{x!}\log(x!) \end{align}

This cannot be put in simpler form, but for large $$\lambda$$ it can be shown that it tends to $$-\frac{1}{2} \log(2 \pi e \lambda)$$

Except for the sign, as noted in a comment, this is the entropy of a Poisson distribution, see eg here or the Wikipedia.

BTW, if you look for higher order correction terms, bear in mind that most results in the bibliography imply a base two logarithm (as usual in information theory).