7
$\begingroup$

new user here self-studying some mathematical statistics. I came across this problem and am stuck.

Problem: Suppose that for $i = 1, ... , n$, the positive random variables $X_i$ are independent and each have the cumulative distribution function $G(x|\alpha) = 1 - e^{-\alpha x^2}$, for $x \geq 0, \alpha > 0.$

(i) Find the uniformly minimum-variance unbiased estimator (UMVUE) of $\sqrt{\alpha}$ based on $X_1, ... , X_n$, and explain why or why not the UMVUE obtained here is unique;

(ii) Solve for the Cramer-Rao Lower Bound (CRLB) for the variance of an unbiased estimator of $\sqrt{\alpha}$;

(iii) Find the variance of the UMVUE and determine if the variance of the UMVUE meets the CRLB, or if the CRLB can even be obtained at all.

Here is what I have tried and know:

Well, to start I know that the random variables $X_1, ..., X_n \stackrel{ind}{\sim} R$ with PDF $:= g(x|\alpha) = 2\alpha xe^{-\alpha x^2}$ have a liklihood function equal to $L(\textbf{x}| \alpha) = (2\alpha)^n(\Pi_{i = 1}^{n}x_i)e^{-\Sigma_{i = 1}^{n}\alpha x_i^2}$, which gives a log-liklihood function equal to $\mathscr{L}(x) = \ln(L(\textbf{x}|\alpha) = n\ln(2) + n\ln(\alpha) + \Sigma_{i=1}^{n}\ln(x_i) - \Sigma_{i=1}^{n}\alpha x_i^2$.

Taking the derivative of $\mathscr{L}(x)$ w.r.t $\alpha$ yields

$\mathscr{L}'(x) = \frac{d[\mathscr{L}(x)]}{d\alpha} = \frac{n}{\alpha} - \Sigma_{i=1}^{n}x_i^2$.

And while I'm aware this isn't provided to me, but from looking at this PDF and prior study I know this is that of a Rayleigh distribution (on the Wikipedia page, let $\sigma^2 = \frac{1}{2\alpha}$ and they're equivalent) which tells me it's expected value, but still, computing the expected value yields:

$E[R] = \int_0^\infty \! 2\alpha x^2e^{-\alpha x^2} \, \mathrm{d}x = \frac{\sqrt{\pi}} {2\sqrt{\alpha}}$. We use this value to define $\tau(\alpha)$. That is, let $\tau(\alpha) = \frac{\sqrt{\pi}} {2\sqrt{\alpha}}$

Now, I know from the textbook I'm using (Casella-Berger) that an estimator $W^{*}$ is a UMVUE of $\tau(\theta)$ if it satisfies $E_{\theta}[W^{*}] = \tau(\theta)$ for all $\theta$ and, for any other estimator $W$ with $E_{\theta}[W] = \tau(\theta)$, $Var_{\theta}(W^{*}) \leq Var_{\theta}(W)$.

Additionally, this distribution/PDF is a member of an exponential family, and has the statistic $T(\textbf{X}) = \Sigma_{i = 1}^{n}X_{i}^{2}$ which is a complete and sufficient statistic.

I know the Lehman-Scheffe theorem tells me that "Unbiased Estimators based on complete sufficient statistics are unique," and that I will have to use the Cramer-Rao inequality, but I'm just getting stuck on actually finding the UMVUE. Do I need to find the distribution of $T(\textbf{X})$? The next parts don't seem too bad once I have it, as it seems like I can maybe use Corollary 7.3.15 which deals with Attainment of the CRLB (on page 341 in chapter 7 if you have a copy), but for some reason I'm just getting stuck on actually finding the UMVUE and would be very grateful for some guidance. I feel like I have some of the pieces (or maybe not) and I'm just not seeing how to assemble them together and find the missing info I need. Thanks for taking the time to read this post and consider my question.

$\endgroup$

1 Answer 1

5
$\begingroup$

You are going down the correct path—when you are looking for the UMVUE in a parametric problem, the simplest method in most cases is to use the Lehmann–Scheffé theorem, which says that if you can form an unbiased estimator from a complete sufficient statistic, then that estimator is the unique UMVUE. Now, from your stipulated distribution, you get the likelihood function:

$$\begin{aligned} L_\mathbf{x}(\alpha) &= \prod_{i=1}^n f_X(x_i|\alpha) \\[6pt] &= \prod_{i=1}^n 2 \alpha x_i \exp(-\alpha x_i^2) \\[6pt] &= (2 \alpha)^n \bigg( \prod_{i=1}^n x_i \bigg) \exp \bigg( -\alpha \sum_{i=1}^n x_i^2 \bigg). \\[6pt] \end{aligned}$$

This likelihood function can be decomposed as:

$$L_\mathbf{x}(\alpha) = h(\mathbf{x}) g_\alpha(T(\mathbf{x})),$$

using the sufficient statistic $T(\mathbf{x}) \equiv \sum_{i=1}^n x_i^2$. With some additional work (which I will leave to you) it can be shown that this statistic is complete, so we can use it as a basis for the Lehmann–Scheffé method. This all simply repeats what you have already figured out, but with some clearer presentation.

To actually form an estimator from this complete sufficient statistic, you will generally need to find its distribution, so that you can form an appropriate function of the statistic to obtain an unbiased estimator. Letting $Y_i = X_i^2$ we have $Y_1,...,Y_n \sim \text{IID Exp}(\alpha)$ (where $\alpha$ is the rate parameter), so you then obtain $T(\mathbf{X}) = \sum_{i=1}^n X_i^2 \sim \text{Gamma}(n, \alpha)$. If you have a look at the moments of this distribution, you will see that the expected value is $n/\alpha$, so at the moment the parameter of interest is entering into the expectation in an inverted form. To deal with that, you may be able to form an unbiased estimator of the form:

$$\widehat{\sqrt{\alpha}} \equiv \frac{\text{const}}{\sqrt{T(\mathbf{x})}} \sim \text{InvNakagami}(\text{parameters}).$$

where the estimator has a scaled inverse-Nakagami distribution with some parameters to be determined. With a bit of work, you should be able to find the appropriate parameters for this distribution and the appropriate scaling constant to get an unbiased estimator. Using the Lehmann–Scheffé theorem, we then conclude that this is the unique UMVUE in this problem. Once you have the form of this estimator, and its distribution, it should also be easy to find its variance, and compare this to the Cramer-Rao lower bound.

$\endgroup$
4
  • $\begingroup$ Thank you for your nice and detailed reply! So then since via a simple transformation via the Jacobian method, you get $\hat{\alpha} \equiv \hat{\alpha}(\mathbf{x}) \equiv \frac{n-1}{T(\mathbf{x})}$ is an unbiased estimator for $\alpha$, so then can you just take the square root of $\hat{\alpha}$ and that gets you the unbiased estimator for $\sqrt{\alpha}$? Or is that wrong? It's clear that taking the expectation of $\hat{\alpha}$ yields $\alpha$, I just don't know if I'm allowed to say from this that $\sqrt{\hat{\alpha}}$ is my unbiased estimator for $\alpha$. $\endgroup$ May 3, 2020 at 23:38
  • 1
    $\begingroup$ Sorry, I didn't read your question correctly --- thought you were estimating $\alpha$. I will amend my answer later once I have thought about how to correct this. (And no, taking the square root won't necessarily work. Will get back to you.) $\endgroup$
    – Ben
    May 4, 2020 at 1:29
  • $\begingroup$ It's okay. Thanks! Looking forward to your edit. $\endgroup$ May 4, 2020 at 12:52
  • $\begingroup$ I have now updated the answer to deal with estimation of $\sqrt{\alpha}$. This makes it a bit more complicated, but it should not be insurmountable. $\endgroup$
    – Ben
    May 8, 2020 at 2:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.