0
$\begingroup$

I understand that MLE considers the joint probability of the observed data so if for example we are trying to estimate $\mu$ given the normal distribution as a PDF $\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^2}{2\sigma^2}}$ given $X, X=\{x_1,x_2,\cdots,x_n\}$ we then consider the joint probability as $P\{X\} = \prod_i^n P\{x_i\} = \prod_i^n \frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x_i-\mu)^2}{2\sigma^2}}$. The we use the log, set the equation $=0$ and solve. But I have always wondered why the integral is not at all present in the formulation. Perhaps I have written it wrong but what am I not understanding.

$\endgroup$
  • 1
    $\begingroup$ The joint probability of observing $(X_1, X_2,\ldots, X_n) = (x_1,x_2,\ldots, x_n)$ is $0$. Instead, the value of the likelihood is defined to be the joint density of $(X_1, X_2,\ldots, X_n)$ evaluated at $(x_1,x_2,\ldots, x_n)$, and considered as a function of the parameter $\mu$. That is, $$L(\mu; x_1,x_2,\ldots,x_n) = f(x_1,x_2,\ldots, x_n; \mu).$$ And for heaven's sake don't murmur the usual shibboleths of "take logs, set derivative$=0$" etc. $L(\mu)$ is in the form of a normal density function (times a constant) and has a maximum at the mean of this normal density function. $\endgroup$ – Dilip Sarwate Sep 13 '15 at 22:05
0
$\begingroup$

First let me touch up a few things in your description of MLE. On the left hand side what we have is $P(X | \mu)$. This is important because we are thinking about this as a function of $\mu$.

Then we take a log (often) because it makes the expression easier to deal with because it breaks the product into a sum and brings down powers of exponential a where parameters (such as $\mu$ often reside). The reason we can do this is that log is monotone increasing so $log(f(x))$ is maximized at the same $x$ as $f(x)$.

We do not set this equal to zero. We take the derivative and set it equal to zero as we do in any unconstrained smooth optimization problem to find critical points.

The reason that I go into this detail is that if you understand why we do what we do, you may understand why we don't do what we don't do. Integration typically means one of two things. Finding the expected value of a function of a random variable (don't need it) or marginalizing out a variable from a joint distribution (don't need it).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.