3
$\begingroup$

In the process of finding unknown parameters $\mu$(mean) and $\sigma^2$(variance) for the maximum likelihood function for Gaussian distribution, we arrive at the following equation.

$\begin{align*} &\ln\left [ \bf p\left ( x | \mu,\sigma ^2 \right ) \right ] = -\frac{N}{2}\ln (2 \pi) - \frac{N}{2}\ln(\sigma ^2) - \frac{1}{2\cdot \sigma ^2} \left \{ \sum_{n=1}^{N}(x_n - \mu)^2 \right \} \\ &\text{where } { \bf x } = \left ( x_1,x_2,x_3 \dots x_N \right )^T \text{ is a data set of observation. } \\ \end{align*}$

Now by differentiating and maximizing with respect to $\mu$ we get

\begin{align*} {\bf \mu_{ml}} = \frac{1}{N}\sum_{n=1}^{N}x_n \\ \end{align*}

Now how to evaluate Expected value of $\bf\mu_{ml}$ ? I am not able to apply summation rule for expected value. I tried $E\left ( \bf \mu_{ml} \right ) = \sum_{??}^{??} \left ( \mu_{ml} \right ) \cdot \left [ p \left ( {\bf x} | \mu_{ml},\sigma^2 \right ) \right ]$.

$\endgroup$
  • 1
    $\begingroup$ The equivalent to summation for continuous distributions is integration. $\endgroup$ – Björn Jul 29 '17 at 6:14
  • $\begingroup$ Thanks ! But what is the domain of integration? $\endgroup$ – Debashish Jul 29 '17 at 6:37
  • 1
    $\begingroup$ What is the domain of the random variable? $\endgroup$ – Björn Jul 29 '17 at 6:51
  • $\begingroup$ OK. That means it is same as the domain of the random variable $x$. It is not explicitly mentioned in the text about the domain of $x$. However the answer to $E(\mu_{ml})$ is given as $\mu$. Could you please help a little bit how they derived? $\endgroup$ – Debashish Jul 29 '17 at 6:57
3
$\begingroup$

$ \begin{align*} &\text{Using} \; \\ &E\left [ X+Y \right ] =E[X]+E[Y] \\ &E[\mu_{\text{ml}}] = \frac{1}{N}\sum_{n = 1}^{N}E(x_n) \end{align*} $

Since each $x_i$ is a Random Variable picked uniformly from a Gaussian distribution, the expected value of each is just the mean of the distribution. That is $E[x_i] = \mu$ for each $x_i$.

Thus $E[\mu_{ml}] = \frac{1}{N}\cdot N \cdot \mu = \mu$

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.