I have 50,000 objects on which I have performed two different types of regression. Using cross validation, I obtained the average $R^2$ score from each model on each of the objects. So now I have a list of 50k pairs of average $R^2$ scores.
I have used a paired Wilcoxon-Mann-Whitney U test and found that their means differ significantly:
> wilcox.test(first$R.2, second$R.2, paired=TRUE) Wilcoxon signed rank test with continuity correction data: first$R.2 and second$R.2 V = 34503366, p-value < 2.2e-16 alternative hypothesis: true location shift is not equal to 0
Is this the right approach? I figure this is the right test because $R^2$ is bounded above by 1 but unbounded below so I can't really make any assumption about the distribution.
How I can report this in my paper? Do I just say that I used the Wilcoxon test and obtained a p-value less than 2.2e-16?
I also wanted to show the mean and median $R^2$s from each of the techniques. But should I just show point estimates of these values, or should I also show SDs and MADs or SEMs or what? Should I bootstrap confidence intervals for them and see whether they overlap?