0
$\begingroup$

I'm trying to show that the mode (i.e. the maximum) of a Gaussian distribution is given by its mean.

Consider that the Gaussian distribution is given by Equation 1: $$N(x|\mu, \sigma^2)=\frac{1}{(2\pi\sigma^2)^{1/2}}exp(-{\frac{1}{2\sigma^2}}(x-\mu)^2)$$

where $\mu$ is the mean and $\sigma^2$ is the variance.

To solve the problem, I know that I should differentiate Equation 1 and set it to zero. Doing so, I can go as far as:

$$exp(-\frac{1}{2\sigma^2}(x-\mu)^2) \times (-\frac{1}{\sigma^2}(x-\mu))\frac{}{} \times \frac{1}{(2\pi\sigma^2)^{1/2}} = 0$$

My question is: how do I solve this equation?

Intuitively, I would say that we must have $x=\mu$ to make one of the terms zero and then, since all the terms are multiplying, we know that $x=\mu$ is a solution of the problem. However, I would like to know how to solve the equation analytically.

$\endgroup$
0
6
$\begingroup$

It's not always wise to jump straight into taking derivatives (indeed there are a number of widely used densities where this will be useless, or if you're sufficiently rash in going about it, may lead you to select values that are not the mode).

Let's think a little first. That often pays handsome dividends.

  1. The location of its mode will be given by the location of the mode of its log -- the height is changed but not which arguments correspond to the largest value. (Strictly monotonic transformation of the density doesn't alter where the modes are)

  2. note further that shifting the function up or down by additive constants and scaling by positive multiplicative constants don't alter the location of a mode

So in the case of the normal, after we take logs, then drop the additive and multiplicative constants, it suffices to find modes of the function $h(x) = -(x-\mu)^2$

Since this is the negative of a perfect square we can identify the (now plainly) unique mode by inspection, though if you feel compelled to use calculus at this stage it's quite straightforward.

Indeed, as whuber notes in comments, we can simply recognize that ($x-\mu$) is shifting the location of the function right by $μ$. That shifts its mode by $μ$, reducing the problem to finding the vertex of the parabola $h(x)=−x^2$, whose location is trivially implied by the fact that $x^2$ is non-negative (& only zero at $x=0$). [In more complicated problems, if all occurrences of $x$ are in the form $(x-\kappa)$ for some $\kappa$, consider $x^*=x-\kappa$, find the mode of the new function of $x^*$ ($x_0^*$, say) and then the original mode is at $x_0=x_0^*+\kappa$.]

Such simple notions often greatly simplify finding the optimum.

$\endgroup$
2
  • $\begingroup$ +1 If you would further add that shifting the location of the function by $\mu$ shifts its mode by $\mu$, you would reduce the problem to finding the vertex of the parabola $h(x)=-x^2$, whose location is implied by the fact that $x^2\ge 0$ for all real numbers. In effect, a sufficient amount of initial thinking reduces the problem to a complete triviality--thereby making your point even more strongly. $\endgroup$
    – whuber
    Sep 19 '17 at 17:13
  • $\begingroup$ @whuber Thanks. Indeed I should have pointed it out explicitly; I was treating it as something people would do by inspection, but in an answer pointing out elementary simplifications like this one, it certainly belongs. $\endgroup$
    – Glen_b
    Sep 19 '17 at 22:21
0
$\begingroup$

Your intuition is good. Removing constants, you have $$exp(-(x-\mu)^2/2\sigma^2) (\mu-x) = 0,$$ which is zero if either $ (x-\mu)^2 \to \infty$ or $\mu = x$.

The last step is to take a second derivative in order to distinguish the maximum from the minimum. Since this is a self-study question I'll leave out the equations, but you can show that at $x=\mu$ the second derivative is negative (corresponding to the maximum) and as $(x-\mu)^2 \to \infty$ the second derivative is positive, corresponding to the minima.

$\endgroup$
5
  • $\begingroup$ Maybe I did something wrong but for the $(x-\mu)^2 \rightarrow \infty$ case I get that the second derivative is zero. $\endgroup$ Sep 14 '17 at 5:08
  • $\begingroup$ It should definitely not be zero. The second derivative is zero only if the function is linear (and this function is exponential). $\endgroup$
    – combo
    Sep 14 '17 at 5:46
  • $\begingroup$ The expression that I have for the second derivative is: $$\frac{1}{\sqrt{2\pi\sigma^2}}\times[exp(-\frac{1}{2\sigma^2}(x-\mu)^2) \times (-\frac{1}{\sigma^2}(x-\mu))^2 + exp(-\frac{1}{2\sigma^2}(x-\mu)^2) \times (-\frac{1}{\sigma^2})]$$ which at $(x-\mu)^2 \rightarrow \infty$ results zero. Is it wrong? $\endgroup$ Sep 15 '17 at 4:35
  • 1
    $\begingroup$ The second derivative certainly can be zero (at isolated points) for nonlinear functions! The point of looking at $|x|\to\infty$ is difficult to fathom, since any mode must occur at a finite value. Also, there is no need to evaluate the second derivative of a continuously differentiable function, defined on all of $\mathbb R$, when there's only one critical point: it must be the unique global maximum. $\endgroup$
    – whuber
    Sep 20 '17 at 0:27
  • $\begingroup$ Fair points, all. I definitely recommend taking Glen_b's approach - I was mostly trying to help the author finish his line of reasoning. $\endgroup$
    – combo
    Sep 20 '17 at 0:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.