0
$\begingroup$

I've a dataset with about 12k values given. It looks like this:

enter image description here

When I try to test for normality I get following results, where p-value is extremely slow:

NormaltestResult(statistic=93.328975616353148, pvalue=5.4183922830109284e-21)
ShapirotestResult(0.9910582304000854, 9.806942512599108e-26)

(values for distribution curve fitting)

Searching best parameters for distribution gennorm (error 1.63000230512e-06)
Searching best parameters for distribution norm (error 2.16497464839e-06)
         sumsquare_error
gennorm         0.000002
norm            0.000002

Am I wrong about thinking it is normally distributed because I get such low p-values?

$\endgroup$
  • 2
    $\begingroup$ If I remember correctly, I think this has been dealt with numerous times on site; I'll try to dig up one of the threads but here's a precis. 1. Your distribution will not be exactly normal. (Ever.) 2. you have a huge sample, so even trivial differences will lead to very low p-values. 3. Testing goodness of fit doesn't tell you about the suitability of using a normal model -- with large samples, very low p-values don't necessarily indicate a problem. $\endgroup$ – Glen_b Jan 23 '16 at 5:23
  • $\begingroup$ Thank you! Please add your comment as answer so I can accept it. Currently I do non-linear regression with other features - non-normality is no problem. $\endgroup$ – x4k3p Jan 23 '16 at 5:26
  • $\begingroup$ If I don't find any of the duplicates I believe are already here, I will make it an answer. $\endgroup$ – Glen_b Jan 23 '16 at 5:31
  • $\begingroup$ This isn't actually a duplicate, but one part of the answer there makes similar points. Still looking for a near-duplicate of this question. $\endgroup$ – Glen_b Jan 23 '16 at 5:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.