Testing normality

Question

I have a large dataset (500000 data, V1 column include all the data).

x <- read.csv("mydata.csv", header=F)
hist(x)

Which gives:

Looking at the data, I believe it is not a normal distribution. As a further check, I constructed a qqplot:

x_norm <- (x$V1 - mean(x$V1))/sd(x$V1)
qqnorm(x_norm); abline(0, 1)

which gave:

To check the goodness of fit of x$V1 (rawdata) to a normal distribution, I used:

rnorm <- rnorm(500000, mean(x$V1), sd(x$V1))
cc <- cbind(rnorm, x$V1)
g <- goodfit(cc, method="MinChisq")
summary(g)

         Goodness-of-fit test for poisson distribution

             X^2 df      P(> X^2)
Pearson 914.5227 17 1.679266e-183
Warning message:
In summary.goodfit(g) : Chi-squared approximation may be incorrect

With plot(g) giving:

Does this seem correct? Can I confidently conclude my dataset X$V1 is or is not a normal distribution?

Based on the above analysis, what other distribution should I test?

Answer 1

I would not rely on p-values for any test of normality (or for much else, frankly). Look at the graphs.

You can, a priori, say that EVERY distribution is non-normal. If you have a large dataset the nonnormality will be statistically significant. The questions are HOW non-normal? Non-normal in what ways? and What are the consequences?

None of these questions is answered by any test of normality or statistical significance.

Why are you testing normality? If it's a test of residuals for some linear model, there was a great quote from George Box ... something like this is "like sending out a rowboat to see if the water is calm enough for an ocean liner"

Answer 2

All I can say is that your eyes are one of your better EDA tools. If your data (with 500,000 observations) doesn't look normal, then there's no reason to even perform a statistical test for normality. Especially with that many data points, any slight deviation from normality should make you fail the test.

It looks like your data is actually discrete, too. You should consider fitting a Binomial or Poisson or some other discrete distribution to the data.