Supposing I have a random variable $X$ with distribution

$$f(x | \boldsymbol{\theta}, \boldsymbol{\varphi}).$$

Here $\boldsymbol{\theta} = (\theta_1,\theta_2,...,\theta_r)$ are $r$ parameters which I want to estimate using maximum likelihood estimation (MLE) and $\boldsymbol{\varphi} = (\varphi_1,\varphi_2,...,\varphi_s)$ are $s$ fixed parameters which are the properties of experimental set-up and are known a-priori. I'm trying to investigate the impact of the wrongly defined $\boldsymbol{\varphi}$ on the results of $\boldsymbol\theta$ estimation.

Let's consider, for example, a model of normally distributed random variable $X$ with mean $\mu_t$ ("$t$" stands for "theoretical") and variance $D$: $$\tag{1} f(x|D,\mu_t) = \frac1{\sqrt{2\pi D}}\exp\left(-\frac{(x-\mu_t)^2}{2D}\right). $$

Here $D \in \boldsymbol{\theta}$ is to be estimated while $\mu_t \in \boldsymbol{\varphi}$ is known and fixed by the experimental set-up. In this case MLE gives the following estimation for the variance: $$\tag{2} \hat{D} = \frac1n\sum_{i=1}^n{(x_i-\mu_t)^2}, $$ where $x_1,x_2,...,x_n$ are experiment results and $n$ is the sample size.

Suppose now that something went wrong during the experiment and $E[X]$ became $\mu_r$ ("$r$" stands for "real") instead of $\mu_t$, but we don't know it's happened so we still estimate $D$ using $\mu_t$. That biases our estimation by $$\tag{3} b(D) = E[\hat{D}] - D = (\mu_t - \mu_r)^2. $$

It is known that MLE is asymtotically unbiased but in this simple case we got the bias which, of course, vanishes when there is no error in experimental set-up, i.e. when $\mu_r = \mu_t$.

So here are my questions:

  1. Do such types of errors always result in the bias of estimated parameter? If so, how it can be proofed?

  2. Here I've used direct calculation of $E[\hat{D}]$ to find the bias. Is there another way to do it?

P. S. It may seems strange to bother with biasing while I could just estimate sample mean and put it in (2) instead of $\mu_t$ and get better results. But it is just the very simple case while in the real task, where maximum likelihood equation is solved using iterative procedure, it can't be done so easily. That's why the answer to the second question is also very important for me.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.