# Is likelihood a conditional probability?

If we have a set of observations $$\mathcal{D} = \{x_i\}_{i=1}^n$$ then the likelihood $$\mathcal{L}$$ is:

$$\mathcal{L}(\theta \mid \mathcal{D}) = P_\theta(\mathcal{D})$$ and if the observations are independent then:

$$\mathcal{L}(\theta \mid \mathcal{D}) = \prod_{i=1}^{n}P_\theta(x_i)$$

where $$P$$ is the probability mass function for a given parameter $$\theta$$.

In maximum likelihood estimation we treat $$\theta$$ as a parameter for estimation. Furthermore, in MAP:

$$P(\theta \mid \mathcal{D}) \propto P(\mathcal{D} \mid \theta) P(\theta)$$

If we assume uniform prior then:

$$P(\theta \mid \mathcal{D}) \propto P(\mathcal{D} \mid \theta)$$

and it is said that MLE and MAP give the same point estimate.

What troubles me is that in Bayesian context the likelihood is $$P(\mathcal{D} \mid \theta)$$ where we treat $$\theta$$ as a random variable so conditional probabilities make sense. But in a Frequentist approach we don't treat the parameters as random variables.

Should we view likelihood as a conditional probability or it depends on the approach (Bayesian vs Frequentist)?

• Not sure to grasp your question. Are you asking about how to interpret the likelihood function? Oct 16, 2022 at 20:11
• The likelihood of $\theta$ given the observations $\{x_i\}_{i=1}^n$ is not a conditional probability for $\theta$ (the suggestion is meaningless in frequentist terms and in general wrong in Bayesian terms - there is no reason to expect it to sum or integrate over $\theta$ to $1$), though it is proportional to the conditional probability $\mathbb P(\{x_i\}_{i=1}^n \mid \theta)$ Oct 16, 2022 at 21:11
• stats.stackexchange.com/questions/2641/… may be helpful Oct 16, 2022 at 21:14

## 1 Answer

Yes, likelihood $$P(X|\theta)$$ is always (technically) a conditional probability. It tells you, how likely the occurrence of the observed data is, given the fixed values of parameters. The usage is drastically different in the two approaches. I agree the notation $$L(\theta|X)$$ can be confusing.

## Role of likelihood for a frequentist

Frequentists work with the likelihood as a loss function. It is used to indicate how well the model with parameters $$\theta$$ fits the data. The goal is to maximize the likelihood, e.g., using EM.

$$\hat{\theta}=\arg\max_\theta P(X|\theta)$$

Because of monotonicity of natural logarithm and its better numerical stability in the interval $$[0,1]$$, you can see log-likelihood used instead.

$$\hat{\theta}=\arg\max_\theta \log P(X|\theta)$$

## Role of likelihood for a bayesian

Bayesian approach thinks of likelihood as an increment of information. Model can get updated by adding new information. E.g., we add five samples to the model. The equations are of course derived from Bayes' theorem.

$$P(\theta|X_{1:5})\propto P(X_{1:5}|\theta)\cdot P(\theta)$$

Now we have our updated model, which has seen the five samples. We can add new information, $$X_6$$.

$$P(\theta|X_{1:6})\propto P(X_{6}|\theta)\cdot P(\theta|X_{1:5})$$