# Bootstrapping Kruskal-Wallis test of significance

I am working with a set of 6 variables collected at 4 time periods (I consider each time period as a class in the following). For each time period, I have 82 397 observations.

If I plot the mean and median values of those variables, I can observe some increase/decrease overtime.

I want to test if those variations are significant (ie: to test if the value observed at one date is significantly different from the value observed at another date).

For that, I am using Kruskal-Wallis non-parametric test since the 6 variables are not normally distributed.

However, regarding the large size of my dataset (329 588 observations divided in four classes), the tests give extremely small p-values, and the null hypothesis is systematically rejected, even if median values are really close between classes.

To get around this problem, I was thinking performing a bootstrapping analysis on randomly designed samples of smaller size (5000 observations in total).

Since I am not statistician, I am not sure of the accuracy of this approach. Would you have any comments or suggestions on what I described? More specifically, would you have any methods/references to evaluate the significance of the test using bootstrapping (I don’t know how to calculate the p-value of bootstrapping in this specific case)?

Thanks for your interest in this question.

• Well put. I would add that computing unitless indexes that don't change with $n$ can be useful, e.g., the $c$-index (concordance probability) between any to groups. Extended box plots stratified by group will also help. – Frank Harrell Aug 5 '15 at 11:58