# Relation between sample standard deviation from data and maximum likelihood estimates

This is my data:-

c(3164, 3362, 4435, 3542, 3578, 4529)


I estimated its sample mean and standard deviation via mean & sd functions and got the following results

mu
 3768.333
sigma
 572.859


now I generated the parameter values that maximize the log likelihood assuming the underlying data is generated from a normal distribution using the following code and optim function

op <- function(para){
loglik <- sum(log(dnorm(x,para,para)))
-loglik
}

optim(c(3200,570),op)

OUTPUT:-

$par  3768.2043 522.9118$value
 46.0705

$counts function gradient 49 NA$convergence
 0

$message NULL  My question is shouldn't the sample sd 572.859 be the parameter value that maximises the log-likelihood rather than 522.9118, and if not then how can we assume that the sample sd is an unbiased estimator of the population sd? • Here the MLE for$\sigma^2$is biased since it divides by$n$, whereas the unbiased estimator for$\sigma^2$divides by$n-1\$. Sep 1 at 10:26

The maximum likelihood estimator to which you refer (for a Gaussian likelihood) can be derived as $$\hat\sigma =\sqrt{ \dfrac{ \sum_{i=1}^N\left( X_i-\bar X\right)^2 }{ N } }$$.

The function you’ve called divides by $$N-1$$. This is because dividing by $$N-1$$ and not taking the square root results in an unbiased estimator of variance, and then the square root of the unbiased estimator of variance is taken as a fairly natural step to estimate the square root of the variance (though this is a biased estimator of standard deviation by Jensen’s inequality).

That is, the sd function is not calculating according to maximum likelihood estimation.

If you increase the sample size, you should see the two results get closer and closer together.

• Does this mean that I can't use MLE to estimate parameter values? And if so how can I estimate it given some data? Sep 2 at 12:44
• Also then for gradient descent methods when can we trust their results if in some cases as above they can give biased estimates of parameter values? Sep 2 at 12:46
• @RishavDhariwal There’s nothing inherently wrong with biased estimators. In fact, many popular estimators (e.g., ridge regression) go out of their way to introduce a bit bias in hopes of scoring major improvements in other estimator properties, such as variance.
– Dave
Sep 2 at 15:30
• I understand that but still I would want to know when applying MLE could introduce bias in my estimates and for which ones should i be careful. For example, here I am getting a biased estimate of SD but other distributions have many different parameters like degree of freedom etc is there any test I can perform on my estimates for this? Sep 3 at 8:09
• @RishavDhariwal “Bias” doesn’t mean the estimator gives you the incorrect value. In fact, you can be basically certain that, no matter how you estimate a parameter, your estimate is incorrect. With that in mind, I do not really understand what you’re asking. Perhaps it would be worth fleshing out your inquiry into a full question to post.
– Dave
Sep 5 at 4:51