Questions tagged [effect-size]

Effect size is "a measure of the strength of a phenomenon or a sample-based estimate of that quantity" [Wikipedia].

Filter by
Sorted by
Tagged with
74 votes
4 answers
6k views

A psychology journal banned p-values and confidence intervals; is it indeed wise to stop using them?

On 25 February 2015, the journal Basic and Applied Social Psychology issued an editorial banning $p$-values and confidence intervals from all future papers. Specifically, they say (formatting and ...
51 votes
1 answer
422k views

How to interpret and report eta squared / partial eta squared in statistically significant and non-significant analyses?

I have data that has eta squared values and partial eta squared values calculated as a measure of effect size for group mean differences. What is the difference between eta squared and partial eta ...
Short Elizabeth's user avatar
46 votes
4 answers
17k views

Are smaller p-values more convincing?

I've been reading up on $p$-values, type 1 error rates, significance levels, power calculations, effect sizes and the Fisher vs Neyman-Pearson debate. This has left me feeling a bit overwhelmed. I ...
Zenit's user avatar
  • 1,846
34 votes
7 answers
96k views

Difference between Cohen's d and Hedges' g for effect size metrics

For an effect size analysis, I am noticing that there are differences between Cohen's d, Hedges's g and Hedges' g*. Are these three metrics normally very similar? What would be a case where they ...
Elpezmuerto's user avatar
  • 1,305
26 votes
4 answers
30k views

What sense does it make to compare p-values to each other?

I have two populations (men and women), each containing $1000$ samples. For each sample I have two properties A & B (first year grade point average, and SAT score). I have used a t-test separately ...
Dov's user avatar
  • 1,810
25 votes
1 answer
13k views

How to use delta method for standard errors of marginal effects?

I am interested in better understanding the delta method for approximating the standard errors of the average marginal effects of a regression model that includes an interaction term. I've looked at ...
Thomas's user avatar
  • 733
25 votes
1 answer
4k views

Is a priori power analysis essentially useless?

I attended a meeting of the Society for Personality and Social Psychology last week where I saw a talk by Uri Simonsohn with the premise that using an a priori power analysis to determine sample size ...
Patrick S. Forscher's user avatar
22 votes
5 answers
3k views

What is effect size... and why is it even useful?

I have an introductory-graduate-level statistics background (assume I know mathematical statistics and probability at an undergraduate level (e.g., Wackerly et al., Ross' Probability), and have some ...
Clarinetist's user avatar
  • 4,957
21 votes
2 answers
55k views

Effect size to Wilcoxon signed rank test?

Some authors (e.g. Pallant, 2007, p. 225; see image below) suggest to calculate the effect size for a Wilcoxon signed rank test by dividing the test statistic by the square root of the number of ...
user avatar
17 votes
4 answers
14k views

What does it mean for a study to be over-powered?

What does it mean for a study to be over-powered? My impression is that it means that your sample sizes are so large that you have the power to detect minuscule effect sizes. These effect sizes are ...
Frank Barry's user avatar
16 votes
2 answers
34k views

Mann-Whitney U-test: confidence interval for effect size

According to Fritz, Morris, and Richler (2011; see below), $r$ can be calculated as an effect size for the Mann-Whitney U-test using the formula $$ r = \frac{z}{\sqrt N} $$ This is convenient to me, ...
grey's user avatar
  • 663
15 votes
5 answers
8k views

Are effect sizes really superior to p-values?

Lots of emphasis is placed on relying on and reporting effect sizes rather than p-values in applied research (e.g. quotes further below). But is it not the case that an effect size just like a p-...
rnorouzian's user avatar
  • 3,986
15 votes
3 answers
25k views

How can I derive effect sizes in lme4 and describe the magnitude of fixed effects?

I have run a mixed effects model with a ln transformed continuous response (seconds) and found a significant effect of the categorical predictor (treatment/control, the only fixed effect in the model)....
JKO's user avatar
  • 775
15 votes
1 answer
1k views

If we shouldn't do post hoc power calculations, are post hoc effect size calculations also invalid?

We have posts on here discussing why post hoc power calculations are flawed. What about post hoc calculations of the effect size that we could catch? That seems logically equivalent. If we are not ...
Dave's user avatar
  • 61.9k
14 votes
1 answer
493 views

Specifying prior for effect size in meta-analysis

My question concerns priors on effect sizes, in my project the measure is Cohen's $D$. Through reading the literature, it seems vague priors are often used, such as in the well-know eight schools ...
D_Williams's user avatar
13 votes
6 answers
2k views

Power paradox: overestimated effect size in low-powered study, but the estimator is unbiased

If we have an underpowered study but manage to reject the null hypothesis, anyway, it makes sense to wonder if we have overestimated the effect size. However, such a concern seems unwarranted if we ...
Dave's user avatar
  • 61.9k
13 votes
1 answer
9k views

Variance of Cohen's $d$ statistic

Cohen’s $d$ is one of the most common ways we measure the size of an effect (see Wikipedia). It simply measures the distance between two means in terms of the pooled standard deviation. How can we ...
JRK's user avatar
  • 613
13 votes
3 answers
303 views

Justification for reporting non-significant effect sizes

We recently received peer review on a manuscript that contained reasonable feedback but also this item: when the p-value is larger than the alpha level chosen (here is .05), it is not necessary to ...
Michael MacAskill's user avatar
13 votes
2 answers
1k views

Understanding Gelman & Carlin "Beyond Power Calculations: ..." (2014)

I am reading Gelman & Carlin "Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors" (2014). I am trying to understand the main idea, the main takeway, but I am confused....
Richard Hardy's user avatar
13 votes
1 answer
980 views

Do likelihood ratios and Bayesian model comparison provide superior & sufficient alternatives to null-hypothesis testing?

In response to a growing body of statisticians and researchers that criticize the utility of null-hypothesis testing (NHT) for science as a cumulative endeavour, the American Psychological Association ...
Mike Lawrence's user avatar
12 votes
2 answers
4k views

What descriptive statistics are not effect sizes?

Wikipedia says effect size is a measure of the strength of a phenomenon or a sample-based estimate of that quantity. An effect size calculated from data is a descriptive statistic that conveys ...
Tim's user avatar
  • 19.4k
12 votes
2 answers
2k views

Can I include an effect size as an independent variable in a meta-regression?

My question is whether I can use an effect size $X$ as a dependent variable and another effect size $Y$ as the independent variable in a meta-regression? For example, I conducted a meta-analysis for ...
Spyros's user avatar
  • 155
12 votes
2 answers
4k views

Effect size for interaction effect in pre-post treatment-control design

If you choose to analyse a pre-post treatment-control design with a continuous dependent variable using a mixed ANOVA, there are various ways of quantifying the effect of being in the treatment group. ...
Jeromy Anglim's user avatar
12 votes
2 answers
502 views

Is there a general definition of the effect size?

The effect-size tag has no wiki. The wikipedia page about the effect size does not provide a precise general definition. And I have never seen a general definition ...
Stéphane Laurent's user avatar
12 votes
2 answers
2k views

Bayesian analysis of contingency tables: How to describe effect size

I'm working through the examples in Kruschke's Doing Bayesian Data Analysis, specifically the Poisson exponential ANOVA in ch. 22, which he presents as an alternative to frequentist chi-square tests ...
Ben's user avatar
  • 749
12 votes
1 answer
2k views

Is the Shapiro Wilk test W an effect size?

I want to avoid misusing normality tests where a large enough sample size will highlight any slight non-normality. I want to be able to say that a distribution is "normal enough". When the population ...
Hugh's user avatar
  • 3,979
11 votes
5 answers
28k views

Calculating eta squared from F and df

I'm trying to compute ANOVA effect sizes from papers that provide an F value without other information. If I understand correctly, the effect size for a single-factor ANOVA is $$ \eta {2} = \frac{ss_{...
octern's user avatar
  • 624
11 votes
2 answers
10k views

Effect size in GLMM

In the output of a GLMM, using a binary variable as response variable and continuous variables as explanatory variables [family = binomial(link="logit")], I obtain, for each variable, an estimate ...
mto23's user avatar
  • 637
11 votes
5 answers
14k views

Omega squared for measure of effect in R?

The statistics book I am reading recommends omega squared to measure the effects of my experiments. I have already proven using a split plot design (mix of within-subjects and between-subjects design) ...
levesque's user avatar
  • 854
11 votes
2 answers
15k views

Calculating effect size for variables in a multiple regression in R

I've been trying to learn about effect size in relation to linear regression and am wondering how to implement it in R. Sure, I have p-values and they indicate how "predictive" the explanatory ...
AMathew's user avatar
  • 1,060
10 votes
5 answers
5k views

Practical significance, especially with percents: "standard" measure and threshold

I, like many people, dislike statistical significance testing. I would much rather measure "practical significance" / effect size. The problem is that I do not know of a "standard" way of doing so. ...
Jessica's user avatar
  • 1,231
10 votes
2 answers
10k views

Can a sample be too large for ANOVA or a t-test?

I have close to a million data sets and whenever I run mean comparison test, either ANOVA or a t-test, I get a significance level of less than .0001 on SPSS. I'm concerned that my sample is so large ...
user24232's user avatar
  • 103
10 votes
2 answers
3k views

In a meta-analysis, how should one handle non-significant studies containing no raw data?

Let's say that I'm conducting a meta-analysis, looking at the performance of group A and group B with respect to a certain construct. Now, some of the studies that I'll come across will report that no ...
Speldosa's user avatar
  • 688
10 votes
3 answers
20k views

Cohen's d for dependent sample t-test

Quick question: I've seen Cohen's d calculated two different ways for a dependent samples t-test (e.g., within-samples design testing the efficacy of a medication with pre/post timepoints). Using the ...
gloryatsea's user avatar
10 votes
2 answers
16k views

Effect size for a one-sample t-test

Imagine I work in a screw factory (which is not true). We have been producing a series of 5cm-long (on average) screws for years. I want to modify the production parameters in order to save some money ...
Remi.b's user avatar
  • 5,102
10 votes
2 answers
526 views

Calculating CIs for $\eta^2$ via Z scores - sample size?

In the thread Confidence Interval for $\eta^2$ it was proposed that if only limited statistics are available (in my case, F, df1, df2, means), one could calculate the 95% CI for $\eta^2$ by: ...
jona's user avatar
  • 1,824
10 votes
2 answers
740 views

How to best handle subscores in a meta-analysis?

I am conducting a meta-analysis of effect sizes d in R using the metafor package. d represents differences in memory scores between patients and healthy. However some studies report only subscores of ...
jokel's user avatar
  • 2,763
10 votes
3 answers
4k views

Is the binomial effect size display (BESD) a misleading representation of effect size?

It's hard for me to accept that Donald Rubin would ever come up with a true lemon of a technique. Yet, that's my perception of the BESD [1, 2, 3]. The original paper by Rosenthal and Rubin (1982) ...
rolando2's user avatar
  • 12.4k
9 votes
3 answers
6k views

Understanding $\chi^{2}$ and Cramér's $V$ results

I have two groups of buyers, A and B, and I want to test whether the difference between the percentage of them who would buy a ...
DroppingOff's user avatar
9 votes
2 answers
2k views

Desired effect size vs. expected effect size

When estimating the effect size for determining optimal sample size for a statistical test, the textbook approach seems to do this from pilot studies or the like. Is it also legal to define effect ...
KarloKatz's user avatar
9 votes
1 answer
989 views

Consequences of exceeding sample size after determination of sample in power analysis

Challenge In our office, I have been involved in a discussion about sample size and its influence on effect size - can you help me out and explain further? Base When conducting a power analysis, ...
Jens's user avatar
  • 563
9 votes
2 answers
8k views

Do Cohen's d and Hedges' g apply to the Welch t-test?

How do you go about calculating effect size for the Welch t-test?
mrchimpbanana's user avatar
9 votes
3 answers
5k views

Calculating effect sizes and standard errors for the difference between two standardized mean differences

I have two related questions, both of which are related to a meta-analysis I am performing where where the primary outcomes are expressed in terms of the standardized mean difference. My studies have ...
Patrick S. Forscher's user avatar
9 votes
2 answers
15k views

Calculating and interpreting effect sizes for interaction terms

Can an effect size be calculated for an interaction effect in general and more specifically using the F-statistic and its associated degrees of freedom? If yes, should this be done and what is the ...
user avatar
9 votes
3 answers
12k views

Linear regression effect sizes when using transformed variables

When performing linear regression, it is often useful to do a transformation such as log-transformation for the dependent variable to achieve better normal distribution conformation. Often it is also ...
user avatar
9 votes
1 answer
2k views

Why is usually the acceptable probability of type 1 and type 2 errors different?

This question is raised by my supervisor and I don't know how to explain. Usually the accepted confident level is 0.95 which means that the probability of type 1 error is 5%. But the usually ...
user2230101's user avatar
9 votes
3 answers
28k views

Reporting coefficient of determination using Spearman's rho

I have two non-normally distributed variables (positively-skewed, exhibiting ceiling effects). I would like to calculate the correlation coefficient between these two variables. Due to the non-...
Lomonosova's user avatar
9 votes
3 answers
1k views

Teaching students about non-significant results and large effect size

This year I am going to teach statistics to sophomore year students of psychology. We'll be training such methods as one-way ANOVA. The example will be the time-reaction of a cognitive task among ...
Lil'Lobster's user avatar
  • 1,478
9 votes
3 answers
3k views

Intraclass correlation and aggregation

Imagine that: You have a sample of 1000 teams each with 10 members. You measured team functioning by asking each team member how well they think their team is functioning using a reliable multi-item ...
Jeromy Anglim's user avatar
9 votes
1 answer
1k views

Does Greenhouse-Geisser correction influence the effect size estimation, the statistical significance treshold, or both of them?

I discussed recently with one of my colleagues about the Greenhouse-Geisser correction for non-sphericity. I told him first that this correction is very conservative and that it would be more ...
Psychokwak's user avatar

1
2 3 4 5
20