Likelihood in Bayes theorem vs in MLE I know that similar questions have already been answered on this platform but none of them were really answering my specific question which is the following:
Bayes' theorem arises solely by rearranging the multiplicative law of probability: 
$$p(\theta|x)p(x) = p(x|\theta)p(\theta)$$
$$ p(\theta|x) = \frac{p(x|\theta)p(\theta)}{p(x)}$$
Hence, all the quantities involved are proper pmfs or pdfs. 
However, I constantly read that the likelihood in Bayes theorem wouldnt be a proper probability (pmf or pdf) since it is not normalized to one. 
How is that possible?
I understand the concept of the likelihood function $L(\theta|x)=p(x|\theta)$ in MLE and why it is not a pdf (or pmf) since it holds the random variable x fixed and varies the parameter $\theta$. However, this cannot be used in Bayes theorem, since Bayes theorem requires that the quantities involved are pdfs (or pmfs) otherwise it would be mathematically wrong. So which mistake am I making or what do I not know about the likelihood in Bayes theorem?
Here 
https://ocw.mit.edu/courses/mathematics/18-05-introduction-to-probability-and-statistics-spring-2014/readings/MIT18_05S14_Reading11.pdf
is a numerical example where the likelihoods indeed do not add up to 1 in Bayes' theorem but I do not understand how this is possible since they should be probabilities and hence should add up to 1.
 A: If integrated the conditional probability you would get
$$ \int_\Theta p(\theta|x)d\theta = 1,$$
as expected - the posterior is a proper probability distribution, where I define proper to be that the integral over the parameter space is 1 and not just finite. But in many cases a probability distribution is in practice a product of bounded, positive functions, each individually not a proper probability distribution. In Bayes' theorem, the posterior is
$$\frac{p(x|\theta)p(\theta)}{p(x)},$$
but this puts no requirements on $p(x|\theta)$ or $p(\theta)$ individually: $p(x|\theta)$ is a probability distribution in $x$, but it is just a function in $\theta$. Thus,
the integral
$$\int_\Theta p(x|\theta) d\theta \neq 1$$
in many interesting cases. 
A: Not sure whether I understand precisely what you don't understand. My impression is that it just confuses you that one can speak about $p(x|\theta)$ as both a "proper pmf/pdf" (if interpreted as function over $x$) and a likelihood (if interpreted as function over $\theta$).
The formula gives you the value for $p(\theta|x)$ for fixed values of $x$ and $\theta$, and for this it doesn't matter whether $p(x|\theta)$ is interpreted as function over $x$ or over $\theta$. So one can say that there are only proper pmfs/pdfs in the formula, but (interpreting differently what $p(x|\theta)$ is a function over) also that there's the likelihood in it, which is not a pdf/pmf. (One can also say that $p(\theta|x)$ and $p(x|\theta)$ are both functions of both $\theta$ and $x$, and again there's some freedom to focus on $x$ or $\theta$ when interpreting them.)
Actually for $p(\theta|x)$ to become a proper pdf/pmf over $\theta$ given $x$, $p(x|\theta)$ must be a pmf/pdf over $x$ for given $\theta$, which is just what it is. It does not have to be a pmf/pdf over $\theta$, which it isn't. 
