1
$\begingroup$

We observe $X_1,\ldots,X_N$ and consider following model: $$ X_i = \theta + w_i\epsilon,\quad \epsilon \sim N(0, 1). $$ Based on above model, we want to estimate $\theta$ given $X_1,\ldots,X_N$ and $w_1, \ldots, w_N$. Let $\bar{X}$ be sample mean of $X_1,\ldots,X_N$. Of course $X_i$ and $\bar{X}$ are members of unbiased estimators of $\theta$.

I'm interested in better estimator (in some sense) than above. Are there anything?

$\endgroup$
1
$\begingroup$

The maximum likelihood estimator will be the weighted mean $X_w = \frac{\sum 1/w_i^2 X_i}{\sum 1/w_i^2}$.

It will be unbiased and will have the smallest variance of $ 1/\sum(1/w_i^2)$, i.e. it is better than a simple mean or any $X_i$

$\endgroup$
  • $\begingroup$ I missed about MLE. Thanks. But How can we confirm the variance minimality of $X_w$? $\endgroup$ – Citrus Nov 25 '16 at 5:34
  • $\begingroup$ The reason is the Cramer-Rao bound en.wikipedia.org/wiki/Cram%C3%A9r%E2%80%93Rao_bound which gives the limit on variance of unbiased estimators. And MLE with the likelihood that is exactly Gaussian gives you a Cramer-Rao bound, i.e. estimator with the smallest possible variance. $\endgroup$ – sega_sai Nov 25 '16 at 10:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.