29 votes
Accepted

Are effect sizes really superior to p-values?

The advice to provide effect sizes rather than P-values is based on a false dichotomy and is silly. Why not present both? Scientific conclusions should be based on a rational assessment of available ...
Michael Lew's user avatar
  • 15.3k
19 votes

Are effect sizes really superior to p-values?

In the context of applied research, effect sizes are necessary for readers to interpret the practical significance (as opposed to statistical significance) of the findings. In general, p-values are ...
Darren James's user avatar
  • 1,231
18 votes

Difference between Cohen's d and Hedges' g for effect size metrics

It seems when people say Cohen's d they mostly mean: $$d = \frac{\bar{x}_1 - \bar{x}_2}{s}$$ Where $s$ is the pooled standard deviation, $$s = \sqrt{\frac{\sum(x_1 - \bar{x}_1)^2 + (x_2 - \bar{x}...
FelixST's user avatar
  • 362
18 votes

Power paradox: overestimated effect size in low-powered study, but the estimator is unbiased

It seems to me that an underpowered study by definition is unlikely to give a small p-value against the null. Consequently, if you do get a small p-value it is likely that you are overestimating the ...
dariober's user avatar
  • 4,260
13 votes

Sample Size impact on Effect size

When your sample size is small, there will be more sampling error attached to your effect size estimates and hence more uncertainty/wider confidence intervals. In that sense, I wouldn't say the effect ...
Christian Geiser's user avatar
11 votes

The value of an Effect Size

There is a growing opinion among statisticians that Cohen's $d$ has more problems than advantages. I recommend that you compute effect estimates in raw data or subject-matter units. Besides losing ...
Frank Harrell's user avatar
10 votes

Do Cohen's d and Hedges' g apply to the Welch t-test?

Depends on what is meant by "effect size" here. Welch's t-test is used to test the null hypothesis $\mu_1 = \mu_2$ when we cannot or don't want to assume that the variances are homoscedastic within ...
Wolfgang's user avatar
  • 17k
10 votes
Accepted

Effect size in GLMM

1) Is the z-value similar to the effect size? No, it is a Wald statistic to test the null hypothesis that the estimate is zero. 2) If not, how can I obtain the effect size for each variable? ...
Robert Long's user avatar
  • 60.9k
10 votes
Accepted

Justification for reporting non-significant effect sizes

A plausible answer: While some researchers prefer hypothesis significance over effect sizes in reporting, it is often beneficial to complement coefficient estimates with effect-size measures, ...
DrJerryTAO's user avatar
  • 1,554
9 votes

Cohen's d for dependent sample t-test

Geoff Cumming has a few comments on the matter (taken from Cumming, 2013): In many cases, however, the best choice of standardizer is not the SD needed to conduct inference on the effect in question. ...
dmartin's user avatar
  • 3,315
9 votes
Accepted

Understanding Gelman & Carlin "Beyond Power Calculations: ..." (2014)

I re-read the paper and this time it seems much clearer. Now also the helpful comments by @Glen_b and @amoeba make lots of sense. The whole discussion is based on a starting point that a ...
Richard Hardy's user avatar
9 votes

Variance of Cohen's $d$ for within subjects designs

There is no know derivation of the variance of $d_{av}$. In fact, this is not how one should compute the $d$ value for a within-subjects design (neither with $d_{rm}$ or the $d_{av}$). There are two ...
Wolfgang's user avatar
  • 17k
9 votes

How can I derive effect sizes in lme4 and describe the magnitude of fixed effects?

The paper suggested by @simone, Brysbaert and Stevens as the title indicates, is focused on 'Power Analysis and Effect Size in Mixed Effects Models', but it includes a calculation of effect size, ...
jordi's user avatar
  • 91
9 votes

Trivial effect size BUT statistically significant?

Yes. For any non-zero effect size as n approaches $\infty$, p approaches 0. This is because for Wald type tests, the test statistic is $\frac{\theta}{s_{\theta}}$, and $s_{\theta}$ typically looks ...
Alexis's user avatar
  • 29.9k
9 votes
Accepted

Fisher's Exact test implementation in R

You have chosen to do a one-sided test and, obviously, order is important in a one-sided test. Your first call to fisher.test is testing the null hypothesis Pct1 = ...
Gordon Smyth's user avatar
  • 12.9k
9 votes
Accepted

Meta-Analyzing Dependent Effect Sizes (Log Response Ratios)--Is Averaging Effects An Acceptable Solution?

Your situation is a case of attempting to meta-analyze dependent effect sizes. I'll outline the various ways one could proceed in answering your first question, and then describe my sense of how one ...
jsakaluk's user avatar
  • 5,514
9 votes

Sample size calculation in COVID-19 study

I know I am several months late, but just want to respond to the other answers. All answers use simulations and/or claim the exact Fisher calculation is too computationally intensive. If you code ...
Peter Calhoun's user avatar
9 votes

Power paradox: overestimated effect size in low-powered study, but the estimator is unbiased

To resolve the issue of bias, note that, when we consider the effect size in a test that rejects, we no longer consider the entire distribution of $\hat\theta$ that estimates $\theta$ but $\hat\theta\...
Dave's user avatar
  • 62.5k
9 votes

Sample Size impact on Effect size

As long as the sample is random, the estimate of the effect size is unbiased even in small samples. However, other things being equal, it will be less precise. It's still your best guess, but the ...
Peter Flom's user avatar
  • 120k
8 votes

The value of an Effect Size

"Cohen's d is defined as the difference between two means divided by a standard deviation for the data" (from https://en.wikipedia.org/wiki/Effect_size#Cohen.27s_d ) Obviously the sign is a matter of ...
Bernhard's user avatar
  • 8,447
8 votes
Accepted

Interpreting Effects Plots in R

The effects-plots (or also the numeric output) give you the predicted values of the outcome for certain given values for the predictors (independent variables). It ...
Daniel's user avatar
  • 1,385
8 votes
Accepted

How to evaluate effect size from a regression output

Maybe an example will be helpful. This very simple example is from Gelman and Hill (2006, p.31-34). We want to predict cognitive test scores of children (kid.score) given their mothers' education (mom....
T.E.G.'s user avatar
  • 2,332
8 votes

How can I derive effect sizes in lme4 and describe the magnitude of fixed effects?

You can indeed compute an effect size in multilevel models. The one provided is called delta total, where total is the total of the variance components. I generally use it when the co-variate in the ...
D_Williams's user avatar
8 votes
Accepted

Interpretation when converting correlation of continuous data to Cohen's d

You've hit on a personal pet peeve of mine. I don't think that the interpretation given in the book (of an r-to-d transformed value of a correlation coefficient that is based on two continuous ...
Wolfgang's user avatar
  • 17k
8 votes
Accepted

Power calculations using pilot effect sizes

This just happens to be a topic that has popped up in a few different areas lately: This interactive tool that accompanies on a pub on the topic: http://pilotpower.table1.org/ This Lakens pre-print: ...
user177051's user avatar
8 votes

P value as a measure of effect size?

You intuition is correct here --- although the p-value is not used as a measure of effect size, you are correct that in some tests, for a fixed sample size the distribution of the p-value is ...
Ben's user avatar
  • 125k
8 votes
Accepted

If we shouldn't do post hoc power calculations, are post hoc effect size calculations also invalid?

The problem is in the use of the "post-hoc effect size," not that its calculation is invalid. A "post-hoc effect size" is fundamentally an estimate of population parameters (e.g., ...
EdM's user avatar
  • 92.5k
8 votes

What does reported "r" mean in the context of a t-test?

Without any other information, I would say $r$ is correlation. And that is an effect size.
Peter Flom's user avatar
  • 120k
8 votes
Accepted

What does reported "r" mean in the context of a t-test?

The reported $r$ is usually referring to an effect size. It is calculated from the t-statistic and the degrees of freedom $$ r = \sqrt{\frac{t^2}{t^2 + df}} $$ where $t$ is the t-statistic and $df$ is ...
Robert Long's user avatar
  • 60.9k
7 votes

Cliff's Delta or Vargha-Delaney A?

Cliff's d and VDA have a lineair relationship. Cliff's d ranges from -1 to 1 and VDA from 0 to 1. VDA = (Cliffs_d + 1)/2. So it does not matter which one you report, you can calculate one from the ...
J.Goedhart's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible