# Maximum Likelihood estimator of population variance and its derivation process

I have 2 questions about maximum likelihood and using it to calculate variance:

Question #1:

The question is about finding the derivative of the score function with respect to the parameter $\sigma^2$, i.e. after taking the log likelihood of the normal function

So we start here $$L= -\frac{n}{2}\log2\pi - \frac{n}{2}\log\sigma^2 - \frac{1}{2 \sigma^2} \sum(x_i - \mu)^2$$ and we get here:

$$\frac{\partial}{\partial \sigma^2} = -\frac{n}{2 \sigma^2} + \frac{1}{2 \sigma^4}\sum(x_i - \mu)^2 =0$$

my question is, why is it $$\sigma^4$$ in other words, why is $\sigma$ raised to the power of 4? Isn't it supposed to be to the power of 3? I know I need to brush over my derivation and specifically, I am predominantly confused about the derivation of logs and I'm working on it, but I would sincerely appreciate it if someone could explain to me why it is raised to the power of 4 during the derivation?

And also, when we get $-\frac{n}{2 \sigma^2}$ after the derivation of $\frac{n}{2}\log \sigma^2$, do we pretty much cancel the log and the power goes into the denominator, am I understanding this correctly?

Question #2: So the basic idea of finding the parameter using this method is by taking the derivative in respect to this parameter?

Thank you!

1. The parameter here is $\sigma^2$, not $\sigma$ (though you could actually do it in terms of $\sigma$ throughout instead, and get to the equivalent final estimator). To reduce your confusion, consider $\tau=\sigma^2$.

$$\frac{d}{d\tau} \tau^{-1} = -\tau^{-2}$$

1. The method is maximum-likelihood. The idea is to find the parameter values that maximize the likelihood function. Under particular circumstances, derivative calculus can be used to find local turning points in the likelihood or more often, the log-likelihood, and to confirm that those turning points are local maxima.

$\hspace 1.3cm$

Sometimes you can produce an argument as to why the local maximum will also be a global maximum (which is what you actually want).

But there are plenty of situations where calculus doesn't solve that problem; don't confuse the tool with the problem it's being applied to. Sometimes you need to find the maximum in other ways.

• Thank you for your answer!But then, why do we need to set this derivative to zero, what's is the purpose of this step, what do we get out of it in terms of estimation? At this point I finally understood the steps and derivation, but I just do not get conceptually the purpose of this whole calculus procedure in terms of parameter estimation. – Jen Oct 8 '14 at 9:34
• – Glen_b Oct 8 '14 at 10:28
• 6. Sufficient conditions for optimality, 7. Calculus of optimization. You might want to try some of the (copious) material on the internet introducing these aspects of calculus. There are videos, web pages, notes, and much more – Glen_b Oct 8 '14 at 10:33
• thank you so much! The picture was incredibly helpful, that's exactly what I was confused about and this visualization helped to understand everything! – Jen Oct 9 '14 at 7:08

1st question: As Glen_b writes, you get $\sigma^4$ precisely because the parameter with respect to which you differentiate in your question is $\sigma^2$. But the log-likelihood may be differentiated also w.r.t. to $\sigma$. Then you get the thir power: $$\frac{d}{d\sigma}=\frac{n}{\sigma}-\sum_{i=1}^n\frac{(x_i-\mu)^2}{\sigma^3}$$.

2nd: If you have a close form of the density and are hence able to calculate the (log)likelihood, you may sometimes find extremes of the function by setting the derivative to zero, which is this case. But also the maximum may be in a point where no derivative exists.

• Thank you! So by setting the derivative to zero, I am able to calculate the extremes of density (if I have a close form of density), am I understanding this correctly? – Jen Oct 8 '14 at 10:44
• Yes... You want to calculate for which parameters the is the joint probability of random variables maximised and setting the derivative of likelihood or log-likelihood (log is monotone function) allows you to do it (sometimes). – DatamineR Oct 8 '14 at 12:35
• That makes sense. I guess because of my deficiencies in Calculus, I am just confused about how finding a critical point visually related to Gaussian of a parameter we are trying to estimate. Is finding a critical point of this log-likelihood function essentially helping us to find/approximate the peak of Gaussian distribution of this parameter we are looking for? Except the log-likelihood is inverse? – Jen Oct 8 '14 at 19:00