I have some questions about the following paragraph which introduces a masters level course.

In this unit we consider the Frequentist (i.e. counting) approach to statistical inference and computing the probability/likelihood of the data $y$ given the parameters $\theta$, $P(y|\theta)$. Under this approach we use Maximum Likelihood Estimation (MLE) to estimate the parameters of probabilistic/likelihood models, i.e. $\hat{\theta} = > arg max_{\theta}P(y|\theta)$. If we are dealing with a known probability distribution of the data then we use $P(y|\theta)$, but in general we are interested in the probability $P(y|x, u, \theta)$ where $u$ can be 'predictor' variables and $x=x(u,\theta)$ can be intermediate variables dependent on $u$ and $\theta$. In this unit we primarily focus on the cases of $P(y|\theta)$ and $P(y|u=x, \theta)=P(y|x, \theta)$.

Q1) I understand that big P means a discrete distribution. Does this statement only apply to discrete distributions?

Q2) Why is it talking about likelihood of data? I understood likelihood to pertain to parameters as in MLE?


1 Answer 1


I think big P in this notation is just for probability in general. That is, $P(y|\theta)$ is just probability of y given $\theta$.

Likelihood is used to refer to the likelihood function, which is the same in form as the probability density/mass function, but with the parameters as the unknown. Thus, while we can have $f(x|\theta)$ as the density/mass function of $X$ given $\theta$ value of parameters, we can have the corresponding likelihood function $L(\theta|x)$, where we are interested in $\theta$ and have the dataset $x$ to use.

  • $\begingroup$ Is likelihood the inverse of probability? I understood that it was not because parameters are not random variables. $\endgroup$
    – Kirsten
    Commented Jan 23, 2021 at 22:06
  • $\begingroup$ The likelihood function is not the inverse of the probability density function. They are exactly the same function in form. The difference is that for the likelihood function, your data (X) is known but your parameters are not. The parameters are not random but they are unknown. By maximizing the likelihood function given your data, you can solve for the parameters as a function of your data. This solution is the MLE. $\endgroup$
    – okobroko
    Commented Jan 23, 2021 at 22:09

Not the answer you're looking for? Browse other questions tagged or ask your own question.