0
$\begingroup$

The question is related to computing the likelihood function of a simple problem for MCMC. The full problem can be found here on page 2.

The question in the title is written more succinctly in the screenshot below: How do we compute $N(\mu|x,\sigma)$ given $\mu$, $\sigma$, $x$?

enter image description here

$\endgroup$
1
  • $\begingroup$ I'm sorry can you clarify what's missing from my question? I don't expect readers to go through the paper. The question is simply: how do we compute the probability a certain $\mu$ is the mean of some normal distribution given a sample x and a SD sigma. $\endgroup$ Jul 27, 2022 at 5:53

1 Answer 1

1
$\begingroup$

Here is the full excerpt in a description of the Metropolis-Hastings algorithm (p.144), when contemplating a move from [current] $\mu=110$ to [proposed] $\mu=108$:

Compare the height of the posterior at the value of the new proposal against the height of the posterior at the most recent sample. Since the target distribution is normal with mean 100 (the value of the single observation) and standard deviation 15, this means comparing $N(100|108, 15)$ against $N(100|110, 15)$. Here, $N(μ|x, σ )$ indicates the normal distribution for the posterior: the probability of value $μ$ given the data $x$ and standard deviation $σ$.

Hence, we are in a situation where the targeted posterior is $$\pi(\mu|x,\sigma)\propto \varphi(x;\mu,\sigma)\propto\exp\{-(x-\mu)^2/2\sigma^2\}$$ as the prior density on $\mu$ is constant. Therefore, $N(μ|x, σ )$ denotes the normal density with mean $x$ and standard deviation $\sigma$. And the Metropolis-Hastings acceptance probability is $$\min\{1,\exp[-(\underbrace{100}_x-\underbrace{108}_{\mu^\text{new}})^2/2\sigma^2+(\underbrace{100}_x-\underbrace{110}_{\mu^\text{old}})^2/2\sigma^2]\}$$ Let me add that the description is very poor, in particular because the author has inverted the roles of $\mu$ and $x$ in the sentence

this means comparing $N(100|108, 15)$ against $N(100|110, 15)$.

which should be

this means comparing $N(108|100, 15)$ against $N(110|100, 15)$.

Of course, mathematically, $N(100|108, 15)=N(108|100, 15)$, so the comparison is correct. But it unnecessarily confuses the reader.

$\endgroup$
3
  • $\begingroup$ In the equation did you mean to use division instead of addition? Otherwise things make sense. It sounds like MCMC is not very useful (at least in this example), since it will basically give you a distribution that has a mean that is (very) close to the first random sample from the target distribution. If the goal is to find the mean (as stated in the paper example). We might as well skip all of the hassle and use that number as the mean. $\endgroup$ Aug 2, 2022 at 2:28
  • 1
    $\begingroup$ The difference is within the exponential function, which equates to a ratio outside the exponential function. $\endgroup$
    – Xi'an
    Aug 2, 2022 at 4:44
  • $\begingroup$ The goal of MCMC is to produce a sample from the posterior, here $N(\mu;100,15)$, rather than simply produce an intuitive estimator like the original observation, here $x=100$. Remember this is a toy example, used to understand the mechanisms of the algorithm. Toy means it is not useful in this specific case. $\endgroup$
    – Xi'an
    Aug 2, 2022 at 4:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.