1
$\begingroup$

I am confused by the visualizations of the likelihood, prior and posterior distribution that I usually see when the Bayes' theorem is explained. An example is the image below:

Visualization of three Gaussian curves for the likelihood, prior and posterior distribution

The x-axis shows the parameter $\theta$ and the y-axis represents the density. The definition of the distributions is the following:

Likelihood = P(Data | $\theta$)
Prior = P($\theta$)
Posterior = P($\theta$ | Data)

Given these definitions, I understand the Prior and Posterior plot (since we are visualizing the distribution of the parameters), but the plot of the likelihood distribution shown above is trickier. I understand the plot of the likelihood distribution where the x-axis shows the data. In the case shown above, if we assume that: (1) the likelihood = 0.8 when $\theta$ = 0.5, (2) the likelihood is Normally distributed and (3) the data is i.i.d, is it correct to say the following?

$$ \prod_i^N \mathcal{N}(x_i \ \ | \ \ \theta=0.5, \sigma) = 0.8$$

$\endgroup$
2
$\begingroup$

Yes, that’s correct.

Remember that the likelihood function varies the parameters, not the data. (See What is the difference between "likelihood" and "probability"?) The likelihood is a function of $\theta$ that assumes $\text{Data}$ is fixed, i.e.

$$ L(\theta \mid \text{Data}) = p(\text{Data} \mid \theta)\text{.} $$

What you have is now three different functions of $\theta$: the prior, the likelihood, and the posterior. They’re able to be plotted on the same axes.

In the example you give, 0.8 is the value of the likelihood function at $\theta=0.5$. In other words, the likelihood of $\theta=0.5$ is 0.8.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.