1
$\begingroup$

I am fairly new to Stats and thus this question. I was going through different materials to learn stats.

  1. Somewhere it's mentioned that a population, if plotted against frequency on a bar graph can produce a normal curve (e.g. popcorn popping, heights of people etc)
  2. Whereas somewhere normal curve is associated with PDF, like it's a function where you put infinite random continuous variables and plot its probabilities to get a normal curve
  3. Whereas somewhere it's written that - use normal curve to take out out inferences out of sample data by using area under the curve and p values and z scores
  4. Whereas somewhere it's written that Binomial distribution follow normal curve when number of trials is huge in number
  5. Whereas somewhere normal curve is associated with central limit theorem

Thus is it used for descriptive stats, or inferential stats? Thus I am not able to get my head around this, that what is a normal curve actually?

Please answer in lay English, without jargon, and as detailed as possible. This will help me get a clearer picture.

$\endgroup$
5
  • $\begingroup$ 5 (of which 4 is a particular example and 1 may be in some circumstances) is the key. $\endgroup$
    – Henry
    Feb 12, 2020 at 8:46
  • $\begingroup$ I have never seen the expression "normal curve" in a formal setting (e.g. textbooks). Apparently Pearson has used it in 1901 (in the paper defining the PCA), and regretted it later (see normal distribution in wikipedia). So I take you refer to the normal distribution? That is, your 2., a (continuous distribution) function. It is not the function itself, but how it is used and interpreted that typically causes confusion (e.g. conflating sample and population, data and theory, fit and inference). $\endgroup$
    – Carsten
    Feb 12, 2020 at 9:04
  • $\begingroup$ if possible, please offer quotes (with references) rather than paraphrased quotes $\endgroup$
    – Glen_b
    Feb 12, 2020 at 11:27
  • $\begingroup$ Thanks @Carsten, so that means that normal distribution is used for both descriptive and inferential stats? Descriptive - when plotting population on a frequency table Inferential - when using central limit theorem and how to get around the confusion as you mentioned in your comments, any way to do that? $\endgroup$
    – Vibhor
    Feb 12, 2020 at 11:51
  • 1
    $\begingroup$ @Vibhor: ReneBt offers a fuller explanation, which has the charm of being close to your question, but disadvantage of being imprecise in mathematical terms (see comment by whuber). Your questions are foundational issues at the heart of statistics and typically disected by good statistics books, inevitably in mathematical language. This is a nuanced area and while sometimes concepts are relatively easy to grasp, they look very convoluted when put into text & math. Confusion can hardly be avoided, but is reduced by training (study) and practice (e.g. simulating your own data and analyses). $\endgroup$
    – Carsten
    Feb 13, 2020 at 7:21

1 Answer 1

3
$\begingroup$

The normal curve, otherwise known as the bell curve or Gaussian, is a mathematical model. If you want the specifics it is below, but this equation is not necessary to understand the rest, just skip to the point by point reply.

$$ f(x) = ae^{-(x-b)^2 / 2c^2} $$

a is a scalar multiplier, b is the central position and c is the spread or standard deviation. It has the property that the result increases faster and faster until it reaches a central value then declines slower and slower. It is unbounded and the symmetric.

In statistics the Gaussian is standardised to give an integral to 1 (since total probability is 1). This gives gives us a probability distribution curve. Since a Gaussian is symmetric the central position b is the mean of the distribution.

So to your points

  1. where its mentiond that a population, if plotted against frequency on a bar graph can produce a normal curve (e.g popcorn popping, heights of people etc)

Strictly speaking this is not true which is the whole point of statistics. Real data can never give a true normal distribution. Measurement errors, finite sampling, skews, machine precision, bounds and many more mean we will never get a perfect normal distribution. However, we can compare the distribution obtained to what we would expect to occur under an ideal normal distribution and determine how closely the data matches (this is called a goodness-of-fit test). This allows us to quantify how well the observed data matches or deviates from the normal distribution. There is a popular saying that 'all models are wrong. some are useful' and this is very much the case with the normal curve - it is wrong for real data but it if very often still close enough to be useful.

  1. Whereas somewhere normal curve is associated with PDF, like its a function where you put infinite random continous variables and plot its probabilities to get a normal curve

Yes. If you want to understand how this works, see above discussion about how Gaussian is converted to PDF.

  1. Whereas somewhere its written that - use normal curve to take out out inferences out of sample data by using area under the curve and p values and z scores

The normal curve is indeed used in making inferences out of the data. This is because the normal curve is a very nice mathematical model with nice properties, but this does lead to an important point that is critical to understanding how to use statistics and to interpret them.

My original version of this answer was heavily influenced by goodness of fit but without explicitly doing so which made the answer misleading. Statistical tests can be considered a mathematical transformation of multiple samples in a dataset into an overall summary that gives us insight into the overall properties of the data.

The normality assumption is relevant at the point where the statistic is converted into probability. If the distribution of the test statistic is close to a normal distribution then we can use the normal curve to make simplifications in our calculation of that probability. The normal curve is smooth and differentiable making it very useful for integrating probabilities. We know the raw test statistic includes noise, so if the normal distribution is a good fit for that statistic then projecting the idealised curve onto the data allows smoothing of our probability estimation. The smoothness means that how the p-value varies against the test statistic is consistent and can easily be interpolated for other points. We can easily calculate what values of the statistic are associated with 95 % or 65 % probabilities by simple scaling.

The probabilities of the raw statistic in contrast will jump around with the noise in the data and not be as smooth. Because the raw statistic is not smooth we can't just rescale the value at 65% probability to calculate what the value will be at 95% probability, we would need to recalculate. So we project the model of normality onto the data and use that projection to infer what is occurring in the data.

Note that it is very clear that the relevance of the normality assumption is critical to whether the resulting probabilities are meaningful. Good practice includes verifying if the normal distribution is sufficiently close to the observed statistic distribution for inference to be reliable and meaningful. When we use assumptions of what the distribution of the statistic will look like this is called 'parametric' because we are assuming some parameters apply that can simplify the analysis.

  1. Whereas somewhere its written that Binomial distribution follow normal curve when number os trials are huge in number

is a special case of

  1. Whereas somewhere normal curve is assosciated with central limit theorem

The central limit theorem is covered extensively in on CV and it is stating that if independent trials are repeated many times on data then estimates of the mean derived from the replicated trials will tend towards a normal distribution for many distribution types. As you requested no jargon, so I leave the details of what impacts relevance in whuber's comments, suffice to say there are many caveats are there are many examples where it is not known to be applicable. This assumes that the errors are independent between trials, for many cases the CLT collapses if there is any correlation between trials since this can introduce a biased sampling that draws the means towards values that recur across trials. CLT should not be interpreted that any distribution can be fitted by a normal curve if numbers are large enough.

Thus is it used for descriptive stats, or inferential stats?

Remember that the normal curve is simply a mathematical model, it is completely use-agnostic. It is therefore not incompatible that it has relevance for each, but it has a different relevance.

Descriptive statistics don't directly depend on distributional assumptions, these are mathematical operations that convert the variation in the data into a summary value. However, whether a particular descriptive statistic is the most appropriate for insight may depend on the distribution of the data. For example income data tends to be highly skewed by enormous income for the rich and a lower bound of 0 for poor. In these cases the median is often preferred for conveying 'typical' income.

For inferential tests, the data needs to be close enough to normal distribution for test that assume normality to be applicable.

$\endgroup$
7
  • $\begingroup$ Your comments on (3) and (5) are sufficiently wrong, in general, as to potentially be misleading. The correct assumption for (3) is the approximate Normality of the sampling distribution of a test statistic rather than of the data. The validity of (5) depends on the parameters, estimation procedures, and underlying distributional assumptions. Failing to recognize these important limitations has led many people to believe the CLT is applicable in cases where it gives incorrect results. Your final comment is incorrect: descriptive stats are not "tests" and make no distributional assumptions. $\endgroup$
    – whuber
    Feb 12, 2020 at 15:07
  • 1
    $\begingroup$ Thanks for the feedback @whuber, clearly in my effort to keep things simple somethings have not come across correctly, particularly with respect to 5. I'll update some simple mistakes now then I'll have to edit the rest to make clearer when I have time to make a good job of it. $\endgroup$
    – ReneBt
    Feb 13, 2020 at 7:25
  • $\begingroup$ @whuber I've had a go at updating my answer more thoroughly. I hope I've addressed your points. I'm afraid I can't think what was misleading about my original (5) comment, the main difference to your comment is the degree of detail provided. I have changed from 'most' to 'many' and made it bold, to make it even clearer that it is not universally applicable. I don't want to get into too many technical terms, but your comment does provide more detail if the OP wishes to get specific details. $\endgroup$
    – ReneBt
    Feb 16, 2020 at 5:06
  • $\begingroup$ Thank you for working on this, Rene. My issue with the original (5) still applies: it's that "population parameter estimates" is far too general for your assertion to be true. The CLT refers only to estimates that are arithmetic means of data. For instance, estimates of the maximum of a Uniform$(0,\theta)$ distribution typically are not means of data and typically do not have asymptotically Normal distributions. $\endgroup$
    – whuber
    Feb 16, 2020 at 15:36
  • 1
    $\begingroup$ @whuber, useful comments that definitely will be relevant should the OP or a reader encounter relevant situations. Again, rather than put detail in the answer in order to keep it simple and jargon free I refer to your comments for the details. I interpret the main thrust of your concerns as that CLT is not universally relevant and flag up that correlation does not guarantee collapse of the CLT, hence my choice of edit. $\endgroup$
    – ReneBt
    Feb 17, 2020 at 9:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.