# Help understanding the results of the Lilliefors test

I have a set of annual rainfall data for Thailand which is gridded, so I have approximately 30x18 grid squares. I am trying to test whether the gamma distribution is suitable for my data, so I am doing the Lilliefors test on the rainfall data from each grid square. I appear to have implemented this fine in Matlab. The problem I am having is trying to understand my results. Here is an example from one grid square:

Test statistic = 0.0782
Critical value (1% significance level) = 0.0958
Critical value (5% significance level) = 0.0811
Critical value (10% significance level) = 0.0738


So from this, I think I reject the null hypothesis at the 10% level, but not at the 1% & 5% levels (as the null hypothesis is rejected if the test statistic exceeds the critical value).

Thing is, I don't really understand how I can accept something at the 1% and 5% level but not the 10% level. I am really not a statistician, so I do struggle to get my head around this stuff. I do tests for statistical significance in my work, where something can be statistically significant at the 10% level but not at the 5% or 1% level, so I think this is why I can't understand it being the other way around.

What does my result say about the goodness of fit using the gamma distribution for my data?

• What's "the other way round"? If the rainfall measure really does follow a gamma distribution (that's the null hypothesis) & you decide to say it doesn't when the test statistic (which measures discrepancy of the data with the gamma distibution), is over 0.0738, then you'll be wrong 10% of the time. If that's too often you can choose a more stringent criterion, like it's being over 0.0958, & be wrong only 1% of the time. – Scortchi - Reinstate Monica Oct 15 '14 at 15:35
• It's not your question but I fit gamma distributions and I would never assess them in this way. I would use a quantile-quantile plot and look for systematic deviations from the expected structure. I would also try to think of other distributions that might do better. In this particular case, it is positive that many workers have found gammas useful for rainfall but negative that (a) your data are hardly independent of each other, which may throw off the P-value any way (b) most work seems to have fit gammas to values for specific stations, not to spatially distributed data. – Nick Cox Oct 15 '14 at 15:41
• BTW, the issues discussed here are generally relevant to testing goodness-of-fit to a theoretical distribution; &, if not for you, for others, it might be useful to note that the gamma distribution has a shape parameter & you therefore have to estimate the distribution of the test statistic by bootstrapping as explained here. – Scortchi - Reinstate Monica Oct 15 '14 at 15:47
• @Scortchi Why 'must' in relation to bootstrapping rather than (say) simulation? Lilliefors certainly didn't use the bootstrap. I fear there's something I missed that would make it necessary. – Glen_b Oct 15 '14 at 16:15
• I understand now that you are looking at separate annual series for 540 grid squares, so some of my comments need modification. But I really don't see a difference. If you are willing to look at 540 Lilliefors test results (and it is to be hoped, follow up any strange results individually), then the graphs sound much more fun. You have a computer to automate a loop, group graphs into pages, etc. Conversely, if you want the tests to make automated decisions for you, and not to look at the data, then good luck, but I advise against. – Nick Cox Oct 15 '14 at 16:45

Regarding the interpretation of the "rejection/not rejection" issue:
The $\alpha$% significance level is "the probability that the test will reject a true null hypothesis". So

"Reject at 10% level": we reject the null hypothesis knowing that there is a 10% chance that we are rejecting it wrongly.

"Not reject at 5% level": we do not reject the null hypothesis remarking that "if we were to reject it, there would be a 5% chance of the rejection being wrong".

But wait: what is the meaning of this last phrase, it seems nonsensical: why do we care about the probability of a wrong rejection since we do not reject the null in the first place?

We care, because it is the other way around: the reason of no-rejection is exactly the decision from our part to bear a probability of wrong rejection of only 5% (and not higher). In other words, exactly because we are being very "conservative", very "intolerant" towards the prospect of a wrong rejection, we find ourselves unable to reject the null.

Whether we reject the null hypothesis or not, is not only a matter of the data and the statistical methods we employ: it is as much a matter of a choice we make, a decision we take, about how high a probability of a wrong rejection we want to accept. The "usual" significance levels, 1%, 5%, 10%, do not have any statistical or in general "objective" justification whatsoever (only historical-sociological, and perhaps philosophical such).

• "we reject the null hypothesis knowing that there is a 10% chance that we are rejecting it wrongly" ... if the null were actually true (and the assumptions also held). – Glen_b Oct 15 '14 at 16:13
• @Glen_b Although I think that the "wrongly" in "reject wrongly" necessarily implies "the null is assumed true", indeed, it is better to include the clarification. – Alecos Papadopoulos Oct 15 '14 at 16:35
• Thanks, I think that makes sense. I was planning on using the 5% level as that appears to be used in the literature in my field (and I use that level for my statistical significance work). At that level, I only reject the null hypothesis in 4% of the grid squares. – emmalgale Oct 15 '14 at 16:38
• Good to know it helped. – Alecos Papadopoulos Oct 15 '14 at 16:44
• @Alecos You're right. Sorry, my comment is unnecesary. – Glen_b Oct 15 '14 at 21:20