Since we were taught MLE (Maximum Likelihood Estimation), a number of questions often bothered me. Why does Maximum Likelihood Estimation work ? Why does it always produce almost accurate results while estimating unknown parameters of some specified distribution ?
A serious doubt which I have is about likelihood. Suppose we're a given some data, say $X_1, X_2, \cdots, X_n$ from some distribution , e.g. Poisson or Gamma or Normal (or anything else) with some unknown parameter(s). Now, in order to calculate the MLE for the unknown parameters, we construct the likelihood function at first, which, I guess, is nothing but the probability of choosing that very sample. This logic is completely fine if the samples are coming from a discrete distribution.
But, for continuous distributions, why does this the same method hold ? Then, total probability of choosing that very sample is $0$ of course. So, for continuous distributions, why do we consider the value of the PDF of the specified distribution at the sample points, and multiply them for obtaining the likelihood function ?