Does the logic of "family-wise error" apply to effect size estimation? Background: As I understand, family-wise error refers to the inflation of Type I error when performing multiple hypothesis tests. For example, if I were to perform multiple post-hoc comparisons following an omnibus ANOVA test, then the collection of post-hoc comparisons would be the "family". As the number of tests increases, the risk of Type I error also increases. Hence, family-wise error corrections (like Bonferroni) adjust the alpha criterion in order to account for this Type I error inflation. For example, if I were to perform 6 tests, then I might divide the normative 0.05 alpha by 6 to obtain an adjusted alpha criterion of 0.008, and use this adjusted alpha to determine significance.

Question: Does the family-wise error logic also apply to effect size calculations? If so, are there any common correction procedures like Bonferroni that can be used to adjust effect sizes like eta-squared or Cohen's D? If not, why not?
 A: Penalized maximum likelihood estimation is a good approach that leads to enhanced ability to take a point estimate out of context and have it not be overstated.  For example, if one selects the group whose mean is farthest from the others, the result will be significantly biased, and penalization reduces this bias.  James-Stein estimators also work, and the best of all approaches is a Bayesian hierarchical model because that is one of the few methods that allows full statistical inference to be carried out in the presence of shrinkage.
A: My first thought after reading your question is to use a penalized regression to adjust effect sizes in multiple comparisons. Since an ANOVA is a form of linear regression with multiple categorical variables, one of the penalized regression techniques (such as a lasso, ridge regression, or elastic net) can be applied to adjust the effect sizes of each comparison (= regression coefficients) towards the group mean. Comparisons which are not "statistically significant" result in the effect size being reduced to zero.
A: You generally wouldn't adjust the estimate of the effect size itself. However, you can adjust the widths of the confidence intervals around the effect size, by using a procedure such as Bonferroni to produce "simultaneous confidence intervals."
Using the Bonferroni procedure, the alpha level for each individual test is set at the desired overall alpha level divided by m (where m is the number of tests). Thus, to adjust confidence intervals when the desired overall confidence level is 1 - alpha, simply compute each individual confidence interval at 1 - alpha/m.
Not all multiple comparisons procedures have corresponding procedures for simultaneous confidence intervals like Bonferroni does, but many do.
A: I disagree with the previous response... The protection afforded by a multiple comparison procedure like bonferroni is to limit the likelihood of making an error (in this case a Type I error). An effect size measure should only be computed on a significant result (where the null hypothesis has been rejected) -  in that case you are no longer working in the space of potentially making an error, you are now working in a space of nominal truth - you have to assume the null to be false in order to do the EF calculation meaningfully. So, you should have arrived at that nominal truth conservatively (with a protected alpha) but once you get there you are free to compute away.... 
