Supposing I have a Poisson distributed random variable $X \sim \text{Poiss}(\lambda)$ with a parameter $\lambda$ that could take integer values only. Let $x$ be a single observation of a random variable. I could now get the estimate $\hat{\lambda}$ of the parameter by maximizing the likelihood function: $$ \hat{\lambda} = \arg\max_{\lambda \in \mathbb{N}}{ \log{p(x|\lambda)} }, \tag{1} $$ where $p(x|\lambda)$ is the Poisson PDF. To get the estimator variance $\text{Var}(\hat{\lambda})$ I first calculate the estimator distribution as $$ f(\hat{\lambda}|\lambda_0)=\sum_{x\in\mathbb{N}}{p(x|\lambda_0)\mathbf{1}_{\{\log{p(x|\hat{\lambda})} \geq \log{p(x|\lambda)}, \forall\lambda\in\mathbb{N}\}}}. \tag{2} $$ Here $\lambda_0\in\mathbb{N}$ is the true parameter value and $\mathbf{1}_{\{\cdot\}}$ is equal to 1 if condition in brackets holds and 0 otherwise. Since I computed (2) I can easily get $\text{Var}(\hat{\lambda})$. But things get worse when I try to estimate the parameter using a sample $x_1, x_2, \dots, x_n$ of the size $n$. In this case the estimator is $$ \hat{\lambda} = \arg\max_{\lambda \in \mathbb{N}}{ \sum_{j=1}^{n}{\log{p(x_j|\lambda)}} }. \tag{3} $$ Now the estimator PDF is $$ f(\hat{\lambda}|\lambda_0)=\sum_{x_1,\dots,x_n\in\mathbb{N}}{p(x_1|\lambda_0)\cdot\ldots\cdot p(x_n|\lambda_0)\mathbf{1}_{\{\sum_{j=1}^{n}{\log{p(x_j|\hat{\lambda})}} \geq \sum_{j=1}^{n}{\log{p(x_j|\lambda)}}, \forall\lambda\in\mathbb{N}\}}}. \tag{4} $$ There is no way to compute (4) for large $n$ (about $10^4$ in my case). So my questions are:
- Can I somehow simplify (4) to break the sum into a product?
- Are there some other ways to estimate the variance of the estimator (3)?
Things that I've tried
First of all I've tried to estimate the variance using Cramer-Rao bound by taking the finite difference derivative: $$ \text{Var}(\hat{\lambda}) = \left(n\mathbb{E}\left[(\log{p(x|\lambda_0+1)}-\log{p(x|\lambda_0))^2}\right]\right)^{-1}. \tag{5} $$ As expected, that didn't work quite well: the variance in simulation was lower than this value.
Then I came across a more general Hammersley–Chapman–Robbins bound. One can find that for my case the lower bound is (e.g. see [Dahiya R.C., Commun. Stat. - Theory Methods, 15(3), 709 (1986)]) $$ \text{Var}(\hat{\lambda}) = \left(e^{n/\lambda_0}-1\right)^{-1}. \tag{6} $$ However, this bound turned out to be too tight and was unreachable for any estimator I could find due to exponential decrease of the bound with $n$ (the same problem stated in Dahiya's article).