I have 2 questions about maximum likelihood and using it to calculate variance:

Question #1:

The question is about finding the derivative of the score function with respect to the parameter $ \sigma^2 $, i.e. after taking the log likelihood of the normal function

So we start here $$ L= -\frac{n}{2}\log2\pi - \frac{n}{2}\log\sigma^2 - \frac{1}{2 \sigma^2} \sum(x_i - \mu)^2 $$ and we get here:

$$\frac{\partial}{\partial \sigma^2} = -\frac{n}{2 \sigma^2} + \frac{1}{2 \sigma^4}\sum(x_i - \mu)^2 =0$$

my question is, why is it $$ \sigma^4 $$ in other words, why is $ \sigma $ raised to the power of 4? Isn't it supposed to be to the power of 3? I know I need to brush over my derivation and specifically, I am predominantly confused about the derivation of logs and I'm working on it, but I would sincerely appreciate it if someone could explain to me why it is raised to the power of 4 during the derivation?

And also, when we get $-\frac{n}{2 \sigma^2}$ after the derivation of $\frac{n}{2}\log \sigma^2$, do we pretty much cancel the log and the power goes into the denominator, am I understanding this correctly?

Question #2: So the basic idea of finding the parameter using this method is by taking the derivative in respect to this parameter?

Thank you!

  1. The parameter here is $\sigma^2$, not $\sigma$ (though you could actually do it in terms of $\sigma$ throughout instead, and get to the equivalent final estimator). To reduce your confusion, consider $\tau=\sigma^2$.

$$\frac{d}{d\tau} \tau^{-1} = -\tau^{-2}$$

  1. The method is maximum-likelihood. The idea is to find the parameter values that maximize the likelihood function. Under particular circumstances, derivative calculus can be used to find local turning points in the likelihood or more often, the log-likelihood, and to confirm that those turning points are local maxima.

$\hspace 1.3cm$enter image description here

Sometimes you can produce an argument as to why the local maximum will also be a global maximum (which is what you actually want).

But there are plenty of situations where calculus doesn't solve that problem; don't confuse the tool with the problem it's being applied to. Sometimes you need to find the maximum in other ways.


1st question: As Glen_b writes, you get $\sigma^4$ precisely because the parameter with respect to which you differentiate in your question is $\sigma^2$. But the log-likelihood may be differentiated also w.r.t. to $\sigma$. Then you get the thir power: $$\frac{d}{d\sigma}=\frac{n}{\sigma}-\sum_{i=1}^n\frac{(x_i-\mu)^2}{\sigma^3}$$.

2nd: If you have a close form of the density and are hence able to calculate the (log)likelihood, you may sometimes find extremes of the function by setting the derivative to zero, which is this case. But also the maximum may be in a point where no derivative exists.

  • $\begingroup$ Thank you! So by setting the derivative to zero, I am able to calculate the extremes of density (if I have a close form of density), am I understanding this correctly? $\endgroup$ – Jen Oct 8 '14 at 10:44
  • $\begingroup$ Yes... You want to calculate for which parameters the is the joint probability of random variables maximised and setting the derivative of likelihood or log-likelihood (log is monotone function) allows you to do it (sometimes). $\endgroup$ – DatamineR Oct 8 '14 at 12:35
  • $\begingroup$ That makes sense. I guess because of my deficiencies in Calculus, I am just confused about how finding a critical point visually related to Gaussian of a parameter we are trying to estimate. Is finding a critical point of this log-likelihood function essentially helping us to find/approximate the peak of Gaussian distribution of this parameter we are looking for? Except the log-likelihood is inverse? $\endgroup$ – Jen Oct 8 '14 at 19:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.