I am watching this great lecture by Nando De Freitas.
He establishes the KL divergence by using maximum liklihood estimation.
I do understand the steps from a math standpoint. I just wonder why he wants to measure the similarity between the distributions P(x|theta) and P(x|theta_0).
I also wonder how I can imagine the distribution P(x|theta_0).
As I understand, theta_0 is just the parameter of the bias term.
Why do we even need a distribution for this?