Skip to main content
2 of 2
edited tags
kjetil b halvorsen
  • 82.8k
  • 32
  • 201
  • 663

An lms estimator that is biased?

We have a free falling body, and its motion has parameters $\theta_0$, $\theta_1$ and $\theta_2$.

Its position (in time t) is given by $\theta_0 + \theta_1 * t + \theta_2 * t^2$

The values $\theta_0$ and $\theta_2$ are known, but we want to estimate $\theta_1$.

Unfortunately the measurements $Y_i$ have some noise, so we have $Y_i = W_i + \theta_0 + \theta_1 * t + \theta_2 * t^2$

We assume: $\theta_0$, $\theta_2$ known, $\theta_1 \sim N(0,1)$, $W_i \sim N(0,v^2)$, all of the variables independent

It so happens (as far as I undertstand) that the posterior, $f_{\theta_1|Y}$ is a normal (that is, is a density function on $\theta_1$ that is normal)

We can find the MAP estimate by minimizing an exponent. By finding $\theta_1$ that minimizes $(\theta_1)^2 + (1/v^2) \sum (y_i - \theta-0 - \theta_1* \theta_i - \theta_2*t_2)^2$

And this results the following MAP estimator:

$\hat \theta_1 = \frac{\sum t_i(y_i -\theta_0-\theta_2* t_i^2)}{v^2+\sum t_i^2}$

My problem is: given that $f_{\theta_1|Y}$ is a normal, the MAP estimator is also a LMS estimator.

And this estimator is biased! Its average value, conditioned on $\theta_1$, is not $\theta_1$

$\hat \theta_1 = \frac{\sum t_i(y_i -\theta_0-\theta_2* t_i^2)}{v^2+\sum t_i^2} = \frac{\sum t_i(\theta_1 * t_i +w_i)}{v^2+\sum t_i^2} $

$E[\hat \theta_1] = \frac{\sum t_i(\theta_1 * t_i)}{v^2+\sum t_i^2}$

Is this ok? Intuitively, I'd expect the least mean squares estimator of $\theta_1$ to have an average $\theta_1$. Is this not a theorem? If so, did I do something wrong? If not, what is the intuition on this?

josinalvo
  • 425
  • 2
  • 10