Likelihood of true lambda decreases as I use more samples of a Poisson R.V Can anyone prove the fact that for samples of a Poisson distribution, the log-likelihood of true (real) lambda is decreased as the number of samples is increased? Or even is it true?
I have calculated the log of the likelihood for ground truth lambda = 20, with some generated data with np.random.poisson(lam = 20, size = n).
As I increase n the log-likelihood is decreased.
I wondered why this is happening, so I tried to write log-likelihood, to set n to infinity and justify my observation, mathematically:

 A: This is expected and completely fine.
Your log-likelihood of the model is the sum of the log-probabilities of each observation, given that parameter. (This is because we assume the data are i.i.d.)
$$\log L(\lambda \mid \vec{x}) = \sum_i \log P(x_i \mid \lambda)$$
These summands are all negative. As you add more terms (from more data points), this sum becomes more negative.
This is true, regardless of whether your model $\lambda$ is the true value.
A: There is no problem if the log likelihood goes to negative infinity. This can happen to several statistical models. Consider the following example.
Let $\mathbf{X} = (X_i)_{i=1}^n$ be i.i.d. observations from the one dimensional exponential with density
$$f_{\theta^*}(x) = h(x)e^{-\theta^*T(x) - \psi(\theta)} \quad.$$
Your log likelihood function for the i.i.d. sample then
$$ l(\theta; \mathbf{X}) = \left[\sum_{i=1}^n log(h(x_i))\right] - \theta\left[\sum_{i=1}^n T(x_i)\right] - n\psi(\theta)\quad.$$
Suppose that $H = E[log(h(X_1))]$ exists. Denote by $\bar{H} = \frac{1}{n}\sum_{i=1}^nlog(h(x_i))$ and $\bar{T} = \frac{1}{n}\sum_{i=1}^n T(x_i)$. We have
$$\frac{1}{n} l(\theta; \mathbf{X}) = \bar{H} - \theta\bar{T} - \psi(\theta) \quad.$$
It is possible to prove that $E[T(X_1)] = \psi'(\theta^*)$. By the Strong Law of Large Numbers and Continuous Mapping theorem, it follows that
$$\frac{1}{n} l(\theta; \mathbf{X}) \overset{a.s.}{\longrightarrow} H - \theta\psi'(\theta^*) - \psi(\theta) \quad.$$
If RHS is negative evaluated at $\theta^*$, then $l(\theta; \mathbf{X})$ will diverge to $-\infty$ almost surely for any $\theta$.
As an exercise, you might want to check that these holds for the Poisson distribution.
This might be confusing since the log likelihood is decreasing even for the true parameter. A better perspective is given by the following question: what happens to the log likelihood difference as the sample sizes grow. That is, given the true parameter $\theta^*$ and $\theta \neq \theta^*$, what happens to
$$l(\theta^*;\mathbf{X}) - l(\theta;\mathbf{X}) \quad$$
as $n$ goes to infinite? For well behaved families
$$l(\theta^*;\mathbf{X}) - l(\theta;\mathbf{X}) \overset{a.s.}{\longrightarrow} +\infty \quad,$$
which is what you would expect.
