I wrote an algorithm that scores all potential X per group, so the data will look like this:
1: X1: 0.000001 X2 0.8000 (MAX) X3 0.0003 (CURRENT) 2: X1: 0.999 (MAX) (CURRENT) X2 0.0003 3: X1: 0.0004 (MAX) X2: 0.0003 (CURRENT) etc.. --> 4000 groups
These scores range from
1 and I assume that the scores are distributed normally. Now I want to "test" whether the MAX scoring X I found significantly differs form the CURRENT X per group. I'm not sure if this is even possible to test, but if so how can this be done? For example, I'm pretty sure the MAX is better when the score is 0.8 compared to 0.0003 in group 1, however for group 3 it is hard to say since both score low and their difference is small.
Reaction to comment @glen_b
The scores were scaled afterwards to fall within 0 and 1 (easier to compare), but actually originated from this distribution:
(This plot is simply based on all scores in all groups). I find the scored to be skewed quite a lot towards the < 0 end, perhaps making the normality assumption invalid. However based on the problem we are trying to solve (using my algorithm) I expect the scores to be normally distributed.