I have two samples, one which has 1436 observations where sd=0.0405, mean=0.7776 and skewness=0.032 and the other which has 4956 observations and sd=0.0416, mean=0.7716 and skewness=-0.0897.
Now i am doing a Welch Two Sample t-test in R, but I am wondering whether I should correct for sample size, as my analysis is overpowered now?
I also am looking at effect sizes now, which seem to be nearly 0 for the initial data. This means the tests can be significant, but the result is meaningless. It also has a power of 0.93.
I decided to sample my data (with replacement) and taking only 500 observations from the 2 groups. Now I am getting an effect size of 0.152 and a power of 0.67. Which seems a better result to me.
Also my thesis supervisor suggested that I should correct for multiple testing, but I am still figuring what it exactly is and how I should perform it.
Does anyone have more suggestions on this?