I have three sets of data from some experiments. I fitted each set to different distributions, and each one fits a different distribution. For example, Gamma, Weibull, and Lognormal. If I want to compare these sets what methods would you suggest? I am not sure, but I think I read somewhere that I can use Student t test and ANOVA only if I have the normal distribution. Therefore in my case what would be the approach that I should take? Similar questions were asked in this link and this link but they were not clear for me. Thank you

  • $\begingroup$ How many observations do you have per set? Are the variances of the sets roughly equal? What do you mean when saying the sets fit different distributions? How did you determine those distributions? The fact that a data-set can be fitted to a distribution other than normal doesn't tell you much about its normality. You can force-fit a data-set to any distribution you like. $\endgroup$ – David Ernst Nov 14 '16 at 18:45
  • $\begingroup$ Thank you for asking these questions. 1.Observations >> 5 per set but please do not take this as the the final word of mine. I have sets of 25 too. That is why I asked the general question. 2.No the variances are not equal at all. 3. it means I have a sample of 25 experiment results that for example only weibull distribution fit best rather than normal distribution. 4. I used Matlab or other commercial or online calculators 5.I do not understand why you say this sentence "The fact that a data-set can be fitted to a distribution other than normal doesn't tell you much about its normality. " $\endgroup$ – Silas Nov 14 '16 at 19:06
  • $\begingroup$ Would you describe that sentence more? I did not force fit it. The Q-Q plot was straight for that Weibull distribution but not for the normal so my question is now that I know my data is not normally distributed, how can I compare them? $\endgroup$ – Silas Nov 14 '16 at 19:12
  • $\begingroup$ Then you did not force fit it. Force fitting would be to decide beforehand that you want to use a certain distribution, optimizing the parameters per MLE and never look at a QQ plot. $\endgroup$ – David Ernst Nov 14 '16 at 19:17
  • $\begingroup$ Yes correct, I did not force fit it. I tested different distributions, and the Q-Q plot was best for example for Gamma for my data set1 and Weibull for data set 2 etc. Now that I want to compare data set 1 to 3, what test should I use? I know ANOVA is for normally distributed data. Some articles say that is robust for non normal too but I do not want risk it. $\endgroup$ – Silas Nov 14 '16 at 19:20

ANOVA is quite robust to violations of normality if the variances are roughly equal and the sample sizes are similar as well. Both are not given here which is a problem.

You could use Welch t-tests instead of ANOVA and all those problems are solved. A Welch t-test does not need equal variances or sample size. You would do a t-test for group 1 against 2, 1 against 3 and 2 against 3. For each test, you allocate 1/3 of your alpha level (Bonferroni correction).

However, per central limit theorem, you need 30 samples per group to do t-tests on non-normal data. This remains a problem unless you can have this amount of data.

  • $\begingroup$ Thank you very much. I will look into the Welch t-test as well. Hopefully, the commercial packages like SPSS or Matlab have them. Is there any way you introduce me a method that also works for small sample numbers too? Somtimes experiments are very expensive. Thank you $\endgroup$ – Silas Nov 14 '16 at 19:24
  • $\begingroup$ BTW, Sorry I do not have enough reputation to thumbs up your answer. $\endgroup$ – Silas Nov 14 '16 at 19:26
  • $\begingroup$ Welch t-tests are easy to do in R or matlab. I never had to do small sample non normal myself, so I am not an expert. You need to look into non-parametric tests. They can handle non normality with small sample sizes. But they have other assumptions that need to be satisfied. $\endgroup$ – David Ernst Nov 14 '16 at 19:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.