I have seen multiple times that a normal distribution is fully specified by mean and variance. It is obvious that the third moment is not necessary for a perfect normal distribution as it is 0. I would like to know if mean and variance are sufficient statistic for normal distribution then why do we have positive Kurtosis - 4th moment to describe the tails of the normal distribution?

  • $\begingroup$ Together, empirical mean and empirical variance constitute a minimal sufficient statistic for the Normal distribution, assuming both parameters are unknown. The kurtosis is a function of these two parameters, not an additional parameter of the distribution. $\endgroup$ – Xi'an Apr 21 '20 at 12:53
  • $\begingroup$ Allright. But isnt variance is also a function of the mean parameter and not an additional parameter by the explanation you mentioned? Because both Kurtosis and Variance require mean. $\endgroup$ – GENIVI-LEARNER Apr 21 '20 at 13:13
  • 2
    $\begingroup$ One needs both $\mu$ and $\sigma$ to parameterise a Normal $\mathcal N(\mu,\sigma^2)$ distribution. The higher moments, including skewness and kurtosis, are all expressed in terms of $\mu$ and $\sigma$. $\endgroup$ – Xi'an Apr 21 '20 at 13:28

There's quite a large amount of confusion in this question.

In the first place, most probabilists who are not statisticians have never even heard of the concept of a sufficient statistic but all of them know that a normal distribution is uniquely characterized among the family of normal distributions by its expected value and variance. That is the sense in which the mean and variance are "sufficient" to identify a normal distribution. That is not about what statisticians call sufficient statistics at all; that's an altogether different concept. That latter concept concerns an i.i.d. sample, and no i.i.d. sample is in any way involved in the statement that the mean and the variance characterize a normal distribution within the family of normal distributions. To say that the sample mean and the sample variance constitute a sufficient statistic for the family of normal distributions means that the conditional distribution of the $n$-tuple of observations given the value of the sample mean and the sample variance does not depend on which normal distribution the sample was drawn from, i.e. does not depend on the mean and variance of the distribution.

Now notice that I said "among the family of normal distributions." The mean and variance do not characterize a normal distribution without that or something equivalent to it. In other words, there are many non-normal distributions that have the same mean and the same variance as a particular normal distribution. To say that the mean and the variance are enough to determine a normal distribution means only that they are enough to separate it from other normal distributions.

Next, why shouldn't a normal distribution have higher moments? The $n$th moment of a distribution is just $\operatorname E(X^n)$ where $X$ is a random variable with that distribution. It exists if, and only if, $\operatorname E\left(\left| X^n \right| \right) \text{ (with an absolute value sign)} <+\infty.$ That's all it means.

  • $\begingroup$ Allright I got the gist of it. So we say mean and variance is sufficient statistic to separate one normal distribution from the other. Well now it makes sense. I thought its sufficient as the reason might be that first and second moment (mean and variance) gives us all the information about the population without any loss of information provided population can be perfectly modeled as normal distribution. $\endgroup$ – GENIVI-LEARNER Apr 24 '20 at 14:41
  • $\begingroup$ In my mind i correlated sufficient with the loss of information. So saying sufficient would mean we dont loose any information in the statistic. Just like that in Taylor series, if we are estimating certain polynomial and if we know its quadratic we can truncate the Taylor series in second power (analogous to second moment in statistics) and would retain all information of the polynomial we are estimating. So first and second power of Taylor series is sufficient. $\endgroup$ – GENIVI-LEARNER Apr 24 '20 at 14:44
  • $\begingroup$ In your first comment about, you still seem confused about the meaning of the term "sufficient statistic." The population mean $\mu$ and the population variance $\sigma^2$ are not statistics at all. Look CLOSELY at the definition of "sufficient statistic" in my answer above. $\endgroup$ – Michael Hardy Jul 1 '20 at 16:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.