1
$\begingroup$

As mentioned in the title, I want to derive the Cramer-Rao Lower bound from the Hammersly-Chapman-Robbins lower bound for the variance of a statistic $T$. The statement for the H-C-R lower bound is the following,

Let $\mathbf{X} \sim f_{\theta}(.)$ where $\theta \in \Theta \subseteq \mathbb{R}^k.$ Suppose $T(\mathbf{X})$ is an unbiased estimator of $\tau(\theta)$ where $\tau \colon \Theta \to \mathbb{R}$. Then we have, \begin{equation*} \text{Var}_{\theta}(T) \ge \displaystyle \sup_{\Delta \in \mathcal{H}_{\theta}}\, \displaystyle \frac{[\tau(\theta + \Delta) - \tau(\theta)]^2}{\mathbb{E}_{\theta}\left(\frac{f_{\theta + \Delta}}{f_{\theta}} - 1\right)^2} \end{equation*} where $\mathcal{H}_{\theta} = \{\alpha \in \Theta \colon \text{ support of } f \text{ at } \theta + \alpha \subseteq \text{ support of } f \text{ at } \theta\}$

Now when $k = 1$ and the regularity conditions hold, taking $\Delta \to 0$ gives the following inequality, \begin{equation*} \text{Var}_{\theta}(T) \ge \displaystyle \frac{[\tau'(\theta)]^2}{\mathbb{E}_{\theta} \left( \frac{\partial }{\partial \theta} \log f_{\theta}(\mathbf{X}) \right)^2} \end{equation*} which is exactly the C-R inequality for univariate case.

However, I want to derive the general form of C-R inequality from the H-C-R bound, i.e. when $k > 1$. But, I have not been able to do it. Though, I was able to figure out that we would have to use $\mathbf{0} \in \mathbb{R}^k$ instead of $0$ and $|\Delta|$ to obtain the derivatives, which was obvious anyways, I couldn't get to any expression remotely similar to the C-R inequality. One of the difficulty arises while dealing with the squares. Since for the univariate case, we were able to take the limit inside and as a result got the square of the derivative. While, for the latter case, we cannot take the limit inside, because the derviate in this case would be a vector and we will have the expression containg the square of a vector which is absurd.

I want to know how to derive the C-R inequality in the latter case?

| cite | improve this question | |
$\endgroup$
  • $\begingroup$ your $k$ denotes the dimensionality of the parameter space $\theta$. Are you assuming that the likelihood is a 1 dimensional (e.g. $\mathbb{R}$) valued function? $\endgroup$ – Lucas Roberts Apr 22 at 23:34
  • $\begingroup$ @LucasRoberts When I tried to solve it, yes I assumed likelihood to be 1-dimensional. I am sorry, but can it be anything else too? $\endgroup$ – Sanket Agrawal Apr 23 at 5:50
  • $\begingroup$ no worries I was just trying to get clarification. If your likelihood output is 1 dimensional then I think the only thing you need in your proof with $\theta \in \mathbb{R}^k, k \gt 1$ is to have sufficient regularity conditions for the limit to be continuous and differentiable. e.g. your hypothesis space should be "reasonably well behaved" in some sense, is that not the case for your scenario? $\endgroup$ – Lucas Roberts Apr 23 at 17:26
  • $\begingroup$ also, you're correct that squares in dimensions >1 are different, usually they are represented in a quadratic form, here is a random example of a quadratic form: faculty.math.illinois.edu/~ash/LinearAlg/415Ch9.pdf $\endgroup$ – Lucas Roberts Apr 23 at 17:28
  • $\begingroup$ @LucasRoberts Yes we need regularity conditions. Infact, since C-R inequality is derived assuming the regularity conditions hold, we will need those assumptions here to proceed further. But, the problem is even if I assume those conditions, I am not really able to make progress in the proof. $\endgroup$ – Sanket Agrawal Apr 23 at 18:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.