We have a free falling body, and its motion has parameters $\theta_0$, $\theta_1$ and $\theta_2$.

Its position (in time t) is given by $\theta_0 + \theta_1 * t + \theta_2 * t^2$

The values $\theta_0$ and $\theta_2$ are known, but we want to estimate $\theta_1$.

Unfortunately the measurements $Y_i$ have some noise, so we have $Y_i = W_i + \theta_0 + \theta_1 * t + \theta_2 * t^2$

We assume: $\theta_0$, $\theta_2$ known, $\theta_1 \sim N(0,1)$, $W_i \sim N(0,v^2)$, all of the variables independent

It so happens (as far as I undertstand) that the posterior, $f_{\theta_1|Y}$ is a normal (that is, is a density function on $\theta_1$ that is normal)

We can find the MAP estimate by minimizing an exponent. By finding $\theta_1$ that minimizes $(\theta_1)^2 + (1/v^2) \sum (y_i - \theta-0 - \theta_1* \theta_i - \theta_2*t_2)^2$

And this results the following MAP estimator:

$\hat \theta_1 = \frac{\sum t_i(y_i -\theta_0-\theta_2* t_i^2)}{v^2+\sum t_i^2}$

My problem is: given that $f_{\theta_1|Y}$ is a normal, the MAP estimator is also a LMS estimator.

And this estimator is biased! Its average value, conditioned on $\theta_1$, is not $\theta_1$

$\hat \theta_1 = \frac{\sum t_i(y_i -\theta_0-\theta_2* t_i^2)}{v^2+\sum t_i^2} = \frac{\sum t_i(\theta_1 * t_i +w_i)}{v^2+\sum t_i^2} $

$E[\hat \theta_1] = \frac{\sum t_i(\theta_1 * t_i)}{v^2+\sum t_i^2}$

Is this ok? Intuitively, I'd expect the least mean squares estimator of $\theta_1$ to have an average $\theta_1$. Is this not a theorem? If so, did I do something wrong? If not, what is the intuition on this?

  • 1
    $\begingroup$ Most Bayes estimators are biased, so this is not a surprise. $\endgroup$
    – Xi'an
    Apr 19, 2020 at 10:44

1 Answer 1


As you have a proper prior distribution $\theta_1 \sim N(0,1)$, the MAP estimator will not be the same as the least squares (LMS) estimator. That identity only holds when using the improper uniform prior.

So there is no paradox, you didn't find a biased least squares estimator.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.