Large difference in sample size, high power and small effect size. Is it true significance? I have two samples, one which has 1436 observations where sd=0.0405, mean=0.7776 and skewness=0.032 and the other which has 4956 observations and sd=0.0416, mean=0.7716 and skewness=-0.0897.
Now i am doing a Welch Two Sample t-test in R, but I am wondering whether I should correct for sample size, as my analysis is overpowered now?
EDIT
I also am looking at effect sizes now, which seem to be nearly 0 for the initial data. This means the tests can be significant, but the result is meaningless. It also has a power of 0.93.
I decided to sample my data (with replacement) and taking only 500 observations from the 2 groups. 
Now I am getting an effect size of 0.152 and a power of 0.67. Which seems a better result to me.
Also my thesis supervisor suggested that I should correct for multiple testing, but I am still figuring what it exactly is and how I should perform it.
Does anyone have more suggestions on this?
 A: You are right to be aware of too much power in cases like yours. Usually, samples are scarce and expensive so one can never have enough power. In your case, however, too much power means to find alternatives that are so close to the hypothesis that we simply don't want to find them since they are practically irrelevant.
This gives some hint to a solution. It is called "test of relevance". For a test of relevance one "inflates" the hypothesis from a point to an interval of irrelevance. Now one calculates an usual confidence interval (just take the default t.test in R) and compares if this confidence interval is disjoint from the interval of irrelevance. If yes, you may reject your irrelevance hypothesis. If not --especially if the point estimation itself lies within the interval of irrelevance-- you can't infer anything.
How to choose the irrelevance margin is a matter of discussion with the practitioner. It is similar to choosing the equivalence margin for equivalence tests. Which size of a difference does he find negligible? Sometimes this is hard to answer. 
Note that you may also check if an equivalence test is in fact the right test for you. If you want to show that there is at most a tiny difference, an equivalence test is your way to go. If you still want to reject the hypothesis but also want to keep some scientific diligence and fairness in face of your huge sample sizes, do a test of relevance.
Inequality of variances or sample sizes is no particular problem here. You are already treating it correctly.
