1
$\begingroup$

Suppose that $x\in R^1$, $y\in [-30,30]$ and $P(y|x) \sim N(y; x,\sigma_x)$. If I were to choose $P(x) \sim N(x;0,\sigma_p)$ and observe a single $y$ value, my MAP estimate of $x$ would be $$\frac{\frac{y}{\sigma_x}}{\frac{1}{\sigma_x}+\frac{1}{\sigma_p}}$$ or $$ \frac{y}{1+\frac{\sigma_x}{\sigma_p}}$$ since the denominator is always a constant greater than 1, the estimate is always the $y$ value divided by a constant that biases it towards 0. Is there a way for me to choose a prior such that the MAP estimate would be some form of $y$ that is biased away from 0?

e.g. If $y$ is -5, MAP estimate of $x$ should be a bit more negative than -5. If $y$ is 5, MAP estimate of $x$ should be a bit more positive than 5.

Also, the MAP estimate should have an analytical expression.

Much thanks in advance!! Also, the more options the better!

edit:

  1. This should work for any value of $y$ and the prior needs to be chosen before seeing $y$.
  2. sorry made a mistake,instead of $y\in R^1$, $y\in [-30,30]$. As long as it's bounded by a negative number and a positive number it would work fine.
$\endgroup$
3
  • $\begingroup$ Why do you need such prior? What is the purpose? $\endgroup$
    – Tim
    Commented Jul 19, 2022 at 15:18
  • $\begingroup$ But why do you want the values to be "away from zero"? What do you mean by that? You want them to be as big as possible? Why? $\endgroup$
    – Tim
    Commented Jul 19, 2022 at 15:25
  • $\begingroup$ By away from 0, I meant that the MAP estimate of x should be y multiplied by some constant or values that is greater than 1 regardless of the value of y. Let me know what other information I should include. Thanks! $\endgroup$
    – user363467
    Commented Jul 19, 2022 at 15:31

1 Answer 1

2
$\begingroup$

In general (i.e., for nonzero prior means), the posterior mean in the normal-normal location model is $$ \frac{1/\tau^2}{1/\tau^2+n/\sigma^2}\mu_0+\frac{n/\sigma^2}{1/\tau^2+n/\sigma^2}\bar{x} $$ for a prior mean $\mu_0$, a prior variance $\tau^2$, a population variance (assumed known here) $\sigma^2$ and a sample size $n$ (so notation is slightly different from yours, see https://www2.bcs.rochester.edu/sites/jacobslab/cheat_sheet/bayes_Normal_Normal.pdf.

Hence, the posterior mean is a weighted average of sample mean and prior mean. So if you want to the posterior mean be be further away from zero than the sample mean, you would need to pick $\mu_0>\bar x$ if $\bar x>0$ and $\mu_0<\bar x$ if $\bar x<0$.

But, since you should pick your prior before seeing the data, it is not clear how you would achieve that in general.

$\endgroup$
2
  • $\begingroup$ Indeed, I would need the prior to bias estimates away from 0 regardless of the value of y, so this unfortunately wouldn't work. Thanks though! $\endgroup$
    – user363467
    Commented Jul 19, 2022 at 15:36
  • $\begingroup$ Of course, the larger you pick $|\mu_0|$, the higher the chance that the posterior is affected in the desired fashion. $\endgroup$ Commented Jul 19, 2022 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.