1
$\begingroup$

Here it is the problem: I am supposed to obtain the maximum likelihood estimate of the mean for some normal distribution $\mathcal{N}(\mu,\sigma^{2})$ where $\sigma^{2}$ is known (let it be $\sigma = 1$ for convenience). As far as I have understood, I am asked to generate some data from the corresponding normal distribution (where $\mu$ is also known), apply the Newton-Raphson method to the score function, which gives the sought estimate, and verify how accurate it is.

My question is: am I on the right track? That is to say, should I generate data and compare the estimate with the real value?

By the way, I am new to statistics and I am still learning R language.

Any help is appreciated. Thanks in advance!

$\endgroup$
2
  • $\begingroup$ The MLE in this case has a well-known and simple form. It is unclear why you would use iterative methods such as Newton-Raphson to try to get it. $\endgroup$
    – Ben
    Apr 29, 2019 at 1:01
  • $\begingroup$ I know how to obtain the MLE here. I think the Professor wants us to compare the closed form to its numerical counterpart. $\endgroup$
    – user242554
    Apr 29, 2019 at 1:32

1 Answer 1

1
$\begingroup$

To generate the data in R, you can use the rnorm function: rnorm(n, mean = 0, sd = 1) where n is the number of observations you would like to generate.

To find the mean and variance of a Gaussian distribution by MLE, you do not need to use Newton-Raphson. By calculating the derivatives of the log likelihood function with respect to $\sigma$ and $\mu$, you can find the mean and variance.

Gaussian distribution density: $$N(\mu, \sigma^2) = \frac{1}{\sigma\sqrt{2\pi}} e^{-(y - \mu)^2 / 2\sigma^2},$$

Log-Likelihood of the Gaussian distribution:

$$\ell = \frac{-n}{2} log(2\pi\sigma^2) - \frac{1}{2\sigma^2} \sum_i (x_i - \mu)^2 .$$

Taking the derivative with respect to $\mu$ will let you find the mean: $$\widehat{\mu} = \bar{x} = \sum_i \frac{x_i}{n}$$

Taking the derivative with respect to $\sigma$ will let you find the variance: $$\widehat{\sigma^2} = \frac{1}{n} \sum_i (x_i - \mu)^2 . $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.