0
$\begingroup$

I have two samples X ($N$= 97) and X2 ($N$=4782) drawn from the same population data. I like to test (using statistical visualizations such as normplot and qqplot and hypothesis tests such as jbtest, chi2gof and kstest in matlab) if the data from each sample is normally distributed.

My first data is:

X = [8.13010235400000,13.6713071300000,14.0362434700000,18.4349488200000,26.5650511800000,30.9637565300000,34.3803447200000,40.6012946500000,45,49.3987053500000,58.6713071300000,59.0362434700000,59.0362434700000,59.0362434700000,61.9275130600000,61.9275130600000,63.4349488200000,63.4349488200000,63.4349488200000,63.4349488200000,63.4349488200000,64.4400348300000,71.5650511800000,71.5650511800000,71.5650511800000,71.5650511800000,75.9637565300000,75.9637565300000,75.9637565300000,75.9637565300000,75.9637565300000,75.9637565300000,75.9637565300000,75.9637565300000,75.9637565300000,75.9637565300000,77.4711922900000,77.4711922900000,77.4711922900000,77.4711922900000,77.4711922900000,77.4711922900000,77.4711922900000,77.4711922900000,77.4711922900000,78.6900675300000,90,90,90,90,90,90,90,90,90,90,90,90,90,90,90,93.1798301200000,97.1250163500000,97.7651660200000,102.528807700000,102.528807700000,102.528807700000,102.528807700000,102.528807700000,104.036243500000,104.036243500000,104.036243500000,104.036243500000,104.036243500000,104.036243500000,104.036243500000,105.255118700000,108.434948800000,108.434948800000,108.434948800000,108.434948800000,109.440034800000,116.565051200000,118.072486900000,120.963756500000,127.746805400000,130.601294600000,135,137.489552900000,139.398705400000,139.398705400000,149.036243500000,153.434948800000,159.227745300000,161.565051200000,179.999998800000,180];

The analyses using statistical visualizations in matlab show that the underlying distributions for both samples are normal. However, from the hypothesis tests, the null hypothesis for the first sample is not rejected using the same significance value (except for the chi-square test), but that for the second sample, X2 is completely rejected.

I am now confused as to how to prove my samples are normally distributed and as well come from the same population data. What can I do in this situation?

PS: sample X2is too large for me to post, but if there is any suggestion on how I could show this, then I don’t mind.

enter image description here

EDIT: I have just collated another set of sample (N = 4700) from the same population data wherein the qqplots and cdf comparisons all look good (see new added image). Strangely, the hypothesis tests with jbtest and kstest in Matlab both rejects the null hypothesis. I am now beginning to believe that these hypothesis tests may not be trusted afterall, particularly for real case data.

PS: I couldn't try the Shapiro-Wilks test as Matlab do not have this.

enter image description here

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ Testing each sample separately for normality does not assess whether the samples come from the same population. What question, then, do you really need to address: that of normality or that of a common distribution? $\endgroup$
    – whuber ♦
    Mar 24, 2018 at 19:44
  • $\begingroup$ Thank you @Whuber. I am more interested in answering the former. That is, if the samples are normally distributed. This is because, I am certain they come from the same population data. $\endgroup$
    – oma11
    Mar 24, 2018 at 21:18
  • $\begingroup$ In that case, the results you report ("$\chi^2$ is completely rejected") have rejected that hypothesis of a common Normal distribution. $\endgroup$
    – whuber ♦
    Mar 24, 2018 at 22:25
  • $\begingroup$ Secondary MATLAB implementation detail: don't let the legend (key) occlude any of the data! $\endgroup$
    – Nick Cox
    Mar 25, 2018 at 8:57
  • $\begingroup$ Your latest results are consistent with your earlier ones. (1) The tests are doing what they are supposed to do. You have definite evidence of non-normality in a large sample. (2) The bounds of 0 and 180 bite visibly. A normal distribution with mean and SD for your data discernibly "wants" to overstep those bounds. (3) Let me emphasise an earlier comment If you have to test for normality -- but I really don't see that you have to here. It's like testing to see if a frog is really a toad; you will get high similarity, but as you know it's a frog, there is no point. $\endgroup$
    – Nick Cox
    Mar 25, 2018 at 23:40

2 Answers 2

Reset to default
2
$\begingroup$

I think the attitude here should not be an attempt to "prove" that the data are normally distributed, but simply to check on whether the data are close enough to normal for that to be an adequate approximation for your unstated purposes.

I'd go further than @CroGo and suggest going straight to quantile plots. The comparison with a straight line reference is much easier than comparing two distribution functions with each other when one is a normal (so sigmoid) ogive and the visual challenge is compare an exact and a rough sigmoid curve. (EDIT 2: The posted distribution function plots confirm my prejudice in not clearly showing the limitations of the data.)

Here are a quantile normal plot (normally distributed data points would follow the line) and a spike representation of the distribution.

enter image description here

The quantile plot here for your smaller sample suggests to me that you can't reject a null hypothesis partly because the sample size is rather small. For many purposes the approximation looks fair but not excellent. If you had a theory (physical or other) that says that the distribution should be normal, then it's not well supported. If your interest is just in using techniques that work well if data are approximately normal, then there is no bad news.

But hold it there:

I have labelled the graphs in terms of 0(45)180 because the sharp limit at 180 makes me wonder whether these are bounded measurements in degrees. A look at the detail in the distribution shown as spikes for distinct values seems to support that idea: why else a spike at 90?

Confusion: I should have read these data into my software (non-disclosure: not MATLAB) in double precision. If the difference between 179.999998800000 and 180 is meaningful to you, that was a coarse approximation.

The rejection of the null hypothesis with a much larger sample size is no surprise. That's likely just an indication that you have more information in the larger sample. The same kind of discrepancy for a larger sample size is more likely to qualify as significant at conventional levels. That's how significance tests work, just as 7/10 coin tosses coming up heads could easily be a fluke, but if you get 700000/1 million you really have evidence of bias.

If a graph for the other, larger sample looks similar to that here, it's a similar conclusion. But if your data are really angles, or the equivalent, on [0, 180] or (0, 180], then the normal is at best a dubious reference, as the normal is unbounded and the angles are bounded. But equivalent distributions for bounded data would be likely to look very similar to the normal, so the objection is one of principle.

Question: Is there any sense in which 0 = 180?

Note: If you have to test for normality, chi-square tests belong in a museum and a dedicated test such as Shapiro-Wilk or Doornik-Hansen is preferable to those you mention: that's my impression from my reading of the literature.

EDIT 1: As @whuber rightly points out in a comment, the question of whether the distributions are similar is not the same as that of whether each is normal.

EDIT 2: The quantile plot for the larger sample shows the effects of bounding more clearly. The distribution is normal in the middle as many are, but in principle the normal isn't an appropriate reference for bounded data where the bounds bite. Thus the quantile plots may be useful exploratory devices, but formal tests for normality seem pointless.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you @Nick. You've said quite a lot here and i'm trying to gradually assimilate them all. As Crogro suggested, I have made a comparison between the theoretical and empirical cdfs, as well as made the qqplots for both distributions. Surprisingly, these results show that the larger sample is even more normally distributed than what was suggested by the hypothesis test. I will add these figures just now for everyone to see. $\endgroup$
    – oma11
    Mar 24, 2018 at 21:04
  • $\begingroup$ Again, thank you for suggesting better tests. I will try them out. Also, while still searching through related questions, I found this post: stats.stackexchange.com/questions/2492/… in which someone stated that all normality test should reject the null hypothesis for large enough sample. And from your statement toward the end, you seem to go with that idea... $\endgroup$
    – oma11
    Mar 24, 2018 at 21:25
  • $\begingroup$ And yes! those are angle measurements... $\endgroup$
    – oma11
    Mar 24, 2018 at 21:27
1
$\begingroup$

(Partial answer): For the KS test a good visualization is plot the ECDF curve against the theoretical CDF curve. If your data does come from the distribution then the ECDF curve should closely mirror that of the CDF curve. I'm not a matlab programmer but here is a relevant link.

For distributional fit qq-plots are very similar to that of ECDF plots in that you are comparing theoretical quantities of the distribution against sampled ones. A linear line would be evidence of a decent fit. Again relevant link.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you @CroGo. Your suggestion gave me another perspective entirely. $\endgroup$
    – oma11
    Mar 24, 2018 at 21:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.