Least squared error on two distributions vs Maximum Likelihood estimator Is there any difference in applying MLE vs squared error of two distributions in the following example?: Assume we have a series of points. We can use KDE to create a distribution. Furthermore, we will change the parameters of some our predefined pdf until the squared error difference between them is minimal. Is there a difference in applying this and using MLE to maximize the likelihood of seeing those points in series with some predefined distribution that we referenced in the last sentence? Thanks.
 A: The strict answer is ‘yes’, as the estimation procedures are totally different, so ‘yes there is a difference between least squares and MLE’. The less strict answer is yes, the fits will likely differ by some degree, which may be large or small.
You have framed the problem to try to eliminate the difference between two empirical distributions, using a generating function that seeks to equate them using least squares. So already you assume that the distributions differ minimally.
Next, what you would do is execute a least squares fit and an MLE fit to both, and I assume compare the fit parameters between the two for both methods.
Both estimates would be valid, but I would personally prefer the MLE given that the generating distributions are known.
A: This does not seem to be the same as what is usually called "least squares fit": you compute the LSQ between a parametric model and another already fitted model: the kernel density estimator (assuming that this is what you mean with "KDE"). Note that the KDE depends on the bandwidth, and thus your method is not even well defined.
The MLE, in contrast, directly uses the data points and thus tries to fit a model to the data and not to some other estimation obtained from the data.
It seems to me that these are quite different approaches.
