I know that similar questions have already been answered on this platform but none of them were really answering my specific question which is the following:
Bayes' theorem arises solely by rearranging the multiplicative law of probability: $$p(\theta|x)p(x) = p(x|\theta)p(\theta)$$ $$ p(\theta|x) = \frac{p(x|\theta)p(\theta)}{p(x)}$$
Hence, all the quantities involved are proper pmfs or pdfs. However, I constantly read that the likelihood in Bayes theorem wouldnt be a proper probability (pmf or pdf) since it is not normalized to one. How is that possible?
I understand the concept of the likelihood function $L(\theta|x)=p(x|\theta)$ in MLE and why it is not a pdf (or pmf) since it holds the random variable x fixed and varies the parameter $\theta$. However, this cannot be used in Bayes theorem, since Bayes theorem requires that the quantities involved are pdfs (or pmfs) otherwise it would be mathematically wrong. So which mistake am I making or what do I not know about the likelihood in Bayes theorem?
Here https://ocw.mit.edu/courses/mathematics/18-05-introduction-to-probability-and-statistics-spring-2014/readings/MIT18_05S14_Reading11.pdf is a numerical example where the likelihoods indeed do not add up to 1 in Bayes' theorem but I do not understand how this is possible since they should be probabilities and hence should add up to 1.