2
$\begingroup$

I have a data-set that contains 251 data points. I used minitab software to test for normality and first plotted a histogram which gave the result as under.

enter image description here

The above plot resembles an almost a normal distribution.However when i plot a PP Plot using Anderson-Darling test i get the significance level below 0.005 which makes me reject the null hypothesis i.e the distribution is normal.

enter image description here

Below is the attached PP plot.

Why are the two plots giving different conclusions ? Is my data normal ?

$\endgroup$
  • $\begingroup$ Both plots are showing you essentially the same thing (though the histogram has too few bins which obscures the important detail more than a better choice would). No, your data are plainly not from a normal distribution, there's a clear spike of values at 10 (though that may not matter a great deal in some situations). Why do you care whether the data are from a normal distribution? $\endgroup$ – Glen_b Oct 24 '17 at 1:06
1
$\begingroup$

From both plots, it's clear that you have something very close to normal except for the clump of observations at AFINN = 10.

Why are you testing whether the data are normally distributed? I ask because 1) Many people incorrectly assume that the data must be normally distributed to do linear regression and 2) How much non-normality can be tolerated depends on why you need normality in the first place (if you do need it).

Also, what are AFINN scores and why is there a clump at 10? This looks like a ceiling effect.

$\endgroup$
  • $\begingroup$ The AFINN are the avg sentiment scores of sentiment keywords which are graded on the basis of polarity i.e positive, negative, neutral on a scale of 1-10 We are trying to detect fake reviews out of a sample of 251 reviews based on the logic that observations with AFINN scores close to the mean i.e normally distributed would be tagged as 'Not Fake'. Adding to that , Reviews with AFINN Scores beyond 2 Standard deviations would be tagged as outliers. $\endgroup$ – Deepon GhoseRoy Oct 23 '17 at 12:29
  • 1
    $\begingroup$ I agree with Peter. There is no reason to suppose that a normal distribution is even a natural reference distribution here. Your data are bounded and the upper bound is often attained. That's enough to rule out normality or not? as either an interesting or a useful question, as the answer is just no. Looking at your data is important, but the histogram alone without the superimposed normal is helpful and almost sufficient. $\endgroup$ – Nick Cox Oct 23 '17 at 13:50
  • $\begingroup$ Strictly that's not a PP plot at all. It's a QQ plot, even though that the normal quantiles are labelled by their cumulative probabilities in percent terms. A PP plot has limits of 0 and 1 on both axes. The historic term "probability plot" is still used for quantile-quantile plots, and indeed that carry-over can be confusing. $\endgroup$ – Nick Cox Oct 23 '17 at 13:53
  • 1
    $\begingroup$ From your comment @DeeponGhoseRoy that is NOT a good way to go about this. Please ask a different question with your actual problem (which is in your comment). Detecting fake reviews will be tricky and a simple solution like this isn't right. $\endgroup$ – Peter Flom Oct 24 '17 at 1:23
  • $\begingroup$ @Peter FLom, Sir have added a question as a separate one. Kindly have a look.stats.stackexchange.com/questions/309574/… $\endgroup$ – Deepon GhoseRoy Oct 24 '17 at 4:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.