Here it is the problem: I am supposed to obtain the maximum likelihood estimate of the mean for some normal distribution $\mathcal{N}(\mu,\sigma^{2})$ where $\sigma^{2}$ is known (let it be $\sigma = 1$ for convenience). As far as I have understood, I am asked to generate some data from the corresponding normal distribution (where $\mu$ is also known), apply the Newton-Raphson method to the score function, which gives the sought estimate, and verify how accurate it is.

My question is: am I on the right track? That is to say, should I generate data and compare the estimate with the real value?

By the way, I am new to statistics and I am still learning R language.

Any help is appreciated. Thanks in advance!

  • $\begingroup$ The MLE in this case has a well-known and simple form. It is unclear why you would use iterative methods such as Newton-Raphson to try to get it. $\endgroup$ – Reinstate Monica Apr 29 '19 at 1:01
  • $\begingroup$ I know how to obtain the MLE here. I think the Professor wants us to compare the closed form to its numerical counterpart. $\endgroup$ – user1337 Apr 29 '19 at 1:32

To generate the data in R, you can use the rnorm function: rnorm(n, mean = 0, sd = 1) where n is the number of observations you would like to generate.

To find the mean and variance of a Gaussian distribution by MLE, you do not need to use Newton-Raphson. By calculating the derivatives of the log likelihood function with respect to $\sigma$ and $\mu$, you can find the mean and variance.

Gaussian distribution density: $$N(\mu, \sigma^2) = \frac{1}{\sigma\sqrt{2\pi}} e^{-(y - \mu)^2 / 2\sigma^2},$$

Log-Likelihood of the Gaussian distribution:

$$\ell = \frac{-n}{2} log(2\pi\sigma^2) - \frac{1}{2\sigma^2} \sum_i (x_i - \mu)^2 .$$

Taking the derivative with respect to $\mu$ will let you find the mean: $$\widehat{\mu} = \bar{x} = \sum_i \frac{x_i}{n}$$

Taking the derivative with respect to $\sigma$ will let you find the variance: $$\widehat{\sigma^2} = \frac{1}{n} \sum_i (x_i - \mu)^2 . $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.