Estimator with variance equal to Cramér-Rao lower bound in $N(x_i\theta,1)$-distribution

Question

Let $Y_1,\ldots, Y_n$ be independent and $N(x_i\theta,1)$ distributed, with for each $Y_i$ a mean of $x_i\theta$ for known $x_1,\ldots,x_n$. In a previous section of this exercise I found that the Cramér–Rao lower bound for estimating $\theta$ is equal to $$1\left/\left(\sum_{i=1}^n x_i^2\right) \right.$$ Now I must find an (UMVU) estimator that has a variance equal to this lower bound.

I found the unbiased estimator $$T=\frac{\sum_{i=1}^n Y_i}{\sum_{i=1}^n x_i}$$

Indeed: $$\operatorname{E}[T]=\frac{1}{\sum_{i=1}^n x_i} \cdot \sum_{i=1}^n \operatorname{E}[Y_i]=\frac{1}{\sum_{i=1}^n x_i} \cdot \sum_{i=1}^n x_i\theta = \theta$$

However, if I'm not mistaken, the variance is

$$\mathbb{Var}[T]=\frac{1}{\left(\sum_{i=1}^n x_i \right)^2} \cdot \sum_{i=1}^n\operatorname{Var}[Y_i]=\frac{n}{\left(\sum_{i=1}^n x_i\right)^2}$$

since all the $Y_i$ are independent, and all have variance $1$.

I suppose this variance does not actually equal the required lower bound. Do I have to use another estimator, or is there actually a way to rewrite the variance above so that it's clearly equal to the lower bound? Or did I just make an error in calculation...

Answer 1

The complete sufficient statistic here is actually $\sum_i x_i Y_i$ and not $\sum_i Y_i$. You can see this by writing out the joint distribution in the exponential family form. As it forms a full-rank exponential family, it is easy to see $\sum_i x_i Y_i$ must be complete sufficient.

Hence if a UMVUE does exist, it must be a function of $T(X) = \sum_i x_i Y_i$. Your estimator is not a function of such a $T(X)$, and so cannot be UMVUE.

Now, clearly we have

$$ET(X) = \sum_i x_i^2 \theta$$

and so an unbiaed estimator is

$$\delta(X) = \frac{\sum_i x_i Y_i}{\sum_i x_i^2}$$

Then we have

$$\operatorname{Var}(\delta(X)) = \frac{\sum_i x_i^2 }{\left(\sum_i x_i^2\right)^2}= \frac{1}{\sum_i x_i^2}$$

So that it attains the Cramer–Rao lowerbound.

In short, your mistake was not checking that $\sum_i Y_i$ is actually a complete sufficient statistic.

Answer 2

The first thing to note here is that what you have is a simple linear regression model with a standard normal error term (i.e., having known unit variance). The model can be written as:

$$Y_i = \theta x_i + \varepsilon_i \quad \quad \quad \quad \quad \varepsilon_1,\ldots,\varepsilon_n \sim \text{IID N}(0,1).$$

The most well-known unbiased estimator of the coefficient in a linear regression model is the ordinary least-squares (OLS) estimator. Since your model has a single explanatory variable, and no intercept term, this estimator is:

$$\hat{\theta} = (\mathbf{x}^\text{T} \mathbf{x})^{-1} (\mathbf{x}^\text{T} \mathbf{y}) = \frac{\mathbf{x} \cdot \mathbf{y}}{\|\mathbf{x}\|^2} = \frac{\sum_i x_i y_i}{\sum_i x_i^2}.$$

The OLS estimator also has a well-known form for its variance:

$$\mathbb{V}(\hat{\theta}) = (\mathbf{x}^\text{T} \mathbf{x})^{-1} \cdot \mathbb{V}(\varepsilon_i) = \frac{1}{\|\mathbf{x}\|^2} \times 1 = \frac{1}{\sum_i x_i^2},$$

which is equal to your specification of the Cramér–Rao lower bound.