I have a automated measurement system which measures the volume of particles. In our case it is known that the distribution of the volume particles follow the lognormal distribution. But under some circumstances this assumption can be violated (errors during the image recording etc). These cases should be recorded. My idea is to do an automated normality test on the log-transformed sample values. When the p-value indicates a rejection of the null hypothesis, the user of the system should be informed. The problem is, that the sample size is usually very big (several thousand values) and since the tests are known to be very sensitive for large samples ( i use the Shapiro-Francia test), i came up with the following idea. Instead of using the whole sample, i create x sub samples by randomly picking n values for each sub sample. The test is than applied on the sub samples and mean p-value for all sub samples x is used to test the null hypothesis. Is this from the statistical point of view a possible solution for my problem?
No. The results depend on n. In the extreme case consider n = 2 or n = 3. This will not detect even large deviations from normality in your sample.
One easier option would be just adjust the significance threshold of your normality test, but in practice one should still follow these two steps:
- Consider which deviations from normality would affect your analysis
- Look for that
- Decide if the magnitudes of the deviations (if any) is strong enough that it has to be taken into account
One option would be to just print QQ-Plots for each sample and have someone with experience look at them. If most of them are good and the bad ones really obvious this will take little time. Still another option would be to use methods which are robust to occasional outliers (but this really depends on your problem whether it's applicable).
I don't think any automated procedure is going to suffice here (well, maybe some complex expert system would, but nothing out of the box).
I think you need to use expert knowledge to come up with a rule that will work, then look at each case that is identified by that rule.