# An lms estimator that is biased?

We have a free falling body, and its motion has parameters $$\theta_0$$, $$\theta_1$$ and $$\theta_2$$.

Its position (in time t) is given by $$\theta_0 + \theta_1 * t + \theta_2 * t^2$$

The values $$\theta_0$$ and $$\theta_2$$ are known, but we want to estimate $$\theta_1$$.

Unfortunately the measurements $$Y_i$$ have some noise, so we have $$Y_i = W_i + \theta_0 + \theta_1 * t + \theta_2 * t^2$$

We assume: $$\theta_0$$, $$\theta_2$$ known, $$\theta_1 \sim N(0,1)$$, $$W_i \sim N(0,v^2)$$, all of the variables independent

It so happens (as far as I undertstand) that the posterior, $$f_{\theta_1|Y}$$ is a normal (that is, is a density function on $$\theta_1$$ that is normal)

We can find the MAP estimate by minimizing an exponent. By finding $$\theta_1$$ that minimizes $$(\theta_1)^2 + (1/v^2) \sum (y_i - \theta-0 - \theta_1* \theta_i - \theta_2*t_2)^2$$

And this results the following MAP estimator:

$$\hat \theta_1 = \frac{\sum t_i(y_i -\theta_0-\theta_2* t_i^2)}{v^2+\sum t_i^2}$$

My problem is: given that $$f_{\theta_1|Y}$$ is a normal, the MAP estimator is also a LMS estimator.

And this estimator is biased! Its average value, conditioned on $$\theta_1$$, is not $$\theta_1$$

$$\hat \theta_1 = \frac{\sum t_i(y_i -\theta_0-\theta_2* t_i^2)}{v^2+\sum t_i^2} = \frac{\sum t_i(\theta_1 * t_i +w_i)}{v^2+\sum t_i^2}$$

$$E[\hat \theta_1] = \frac{\sum t_i(\theta_1 * t_i)}{v^2+\sum t_i^2}$$

Is this ok? Intuitively, I'd expect the least mean squares estimator of $$\theta_1$$ to have an average $$\theta_1$$. Is this not a theorem? If so, did I do something wrong? If not, what is the intuition on this?

• Most Bayes estimators are biased, so this is not a surprise. Apr 19, 2020 at 10:44

As you have a proper prior distribution $$\theta_1 \sim N(0,1)$$, the MAP estimator will not be the same as the least squares (LMS) estimator. That identity only holds when using the improper uniform prior.