# Bayes theorem: a function of D, or theta, or both?

In the Bayes formula as written for machine learning applications, $$p(\theta|D) = \frac{ p(D|\theta) p(\theta) }{ p(D) }$$ where $$D$$ is the data, $$\theta$$ are the model parameters.

Commonly $$p(\theta)$$ is labeled the prior, $$p(D|\theta)$$ is called the likelihood, and $$p(D)$$ is called the evidence (or marginal likelihood I think).

The question: I am bothered by calling $$p(D|\theta)$$ as a likelihood. I believe a likelihood is not a probability density, meaning that $$\int p(D|\theta) d\theta$$ does not integrate to one. Wheras I think $$\int p(D|\theta) d\,D$$ does integrate to one.

So it seems strange to include something that is not a probability in the formula. I think maybe my confusion is about whether the overall Bayes formula should be considered as "a function of" $$D$$, or $$\theta$$, or either or both?

In probability theory (as this has nothing specific to machine learning or statistics) Bayes' formula is based on the joint distribution of the pair of random variables $$(D,\theta)$$, $$p(D,\theta)$$, which can be expressed either as $$p(D,\theta)=p(D)\times p(\theta|D)$$ or as $$p(D,\theta)=p(\theta)\times p(D|\theta)$$ [with the confusion notation of using the same $$p(\cdot)$$ everywhere!] where
1. $$p(D)$$ denotes the marginal density of $$D$$ [integrates to one in $$D$$] also called marginal likelihood or evidence
2. $$p(\theta)$$ denotes the marginal density of $$\theta$$ [integrates to one in $$\theta$$] also called prior
3. $$p(D|\theta)$$ denotes the conditional density of $$D$$ given $$\theta$$ [integrates to one in $$D$$] also called likelihood and often denoted $$\ell(\theta)$$
4. $$p(\theta|D)$$ denotes the conditional density of $$\theta$$ given $$D$$ [integrates to one in $$\theta$$] also called posterior
• This seems intuitive. (Typo in #4, "DS"). However I am bothered by calling $p(D|\theta)$ a likelihood if it is viewed as an object where we integrate D, and so it seems to be a function of D rather than of $\theta$. I am sure this comment is stupid, but that is my feeling. – basicidea Nov 29 '18 at 18:09