2
$\begingroup$

The following QQ plot looks with too many points out of the line, the density plot looks normal and the Shapiro Test p-value < 2.2e-16, so this is not a normal distribution but I've read not to trust Shapiro Test when I have about 1000 data points so I should conclude that this distribution is normal?enter image description here

enter image description here

$\endgroup$
  • $\begingroup$ Clear asymmetry suggests something is up. A) Did you calculate the skewness? A nominal rule-of-thumb is if Pearson Skewness $\ge$ 0.1, then you have to take corrective action, e.g., performing statistics on the log() of your measurements, rather thanon the measurements direclty. Also, are there additional factors (meta data, etc.) that you can use to subselect your data? This may also be a mixture of models, so you might have two or 3 normal distributions all sitting close together, but the second and third are small enough so as not to create an obviously multimodal histogram. $\endgroup$ – Peter Leopold Feb 17 at 16:07
  • 1
    $\begingroup$ @Peter Where does that rule of thumb come from? It's not generally applicable, so it would be of interest to know its limitations and assumptions. $\endgroup$ – whuber Feb 17 at 16:12
  • 1
    $\begingroup$ @PeterLeopold "Pearson skewness" is not uniquely defined. Pearson himself put most emphasis on measuring skewness relative to the mode, which had a major role in his system of distributions. But he did also use a dimensionless ratio based on third and second moments around the mean. And yet again (mean $-$ median) / SD appears in his work. But regardless I wouldn't regard skewness of about 0.1 on any measure I've encountered as requiring transformation. I would always want to see the data, however. $\endgroup$ – Nick Cox Feb 17 at 16:18
  • $\begingroup$ @Peter The problem here is more about kurtosis than skewness and it's not clear that taking the log is justified, even if it were skew. It depends on what the OP is going to use the data for. $\endgroup$ – Peter Flom Feb 17 at 16:18
  • $\begingroup$ I would not trust a scale for happiness [NB] with such results! $\endgroup$ – Nick Cox Feb 17 at 16:20
2
$\begingroup$

First, the density plot does not really look normal. It's symmetric, but the shape is wrong. I suggest generating a normal distribution with the same mean and variance as yours and then overlaying that density on the one you've got. I am fairly sure you will see a mismatch.

Second, a quantile normal plot is often a better clue to nonnormality.

Third, and probably most importantly, why are you concerned about the normality of this variable? What are you going to do with the variable?

$\endgroup$
  • 1
    $\begingroup$ Thanks for all comments and answers, @Peter Flom I've got measures of "Happyness" for two groups of people, I plotted them against time and looks like one group gets higher values so I'm trying to statistically compare them, I run Shapiro test for both groups and got p value << 0.05 so I don't know if a t-test or Wilcoxon $\endgroup$ – AnaHochma Feb 17 at 18:42
  • $\begingroup$ I'd go with Wilcoxon. Or maybe a bootstrap. $\endgroup$ – Peter Flom Feb 17 at 19:08
  • $\begingroup$ Ah, two distributions! Thanks for confirming what the data was hinting strongly at. $\endgroup$ – Peter Leopold Feb 17 at 19:15
  • 1
    $\begingroup$ Thanks! so I'll go with Wilcoxon too :) the main reason is because the Shapiro test << 0.05? $\endgroup$ – AnaHochma Feb 17 at 19:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.