Questions tagged [effect-size]
Effect size is "a measure of the strength of a phenomenon or a sample-based estimate of that quantity" [Wikipedia].
966
questions
1
vote
0
answers
161
views
sum of classic eta squared greater than 1
I know that in an ANOVA with multiple factors (let's say x1 and x2), the sum of their respective classic eta squared (not 'partial' eta squared) cannot be greater than one and should approximate the ...
- 103
3
votes
1
answer
1k
views
Is the odds ratio not a measure of effect size?
I have a question regarding the odds ratio. A paper that I submitted was sent back for revision after peer review. However, one of the authors asked a question that has left me quite confused.
In this ...
- 133
0
votes
0
answers
400
views
Converting effect size from linear and logistic regression
I want to calculate a same effect size from a linear regression and a logistic regression to be able to compare them. More specifically, I would like to calculate standardized mean-difference (SMD).
I ...
- 1
1
vote
0
answers
26
views
How do I approach calculating an optimal sample size for injecting mice for triggering an immune response?
Hypothetical: there is a research team tasked with injecting mice aimed at triggering a specific immune response to produce monoclonal antibodies (MAb). Up until now, they've been largely unsuccessful ...
- 11
5
votes
1
answer
216
views
Comparing power between models: how to set a fixed effect size in glm?
I am trying to understand how much power we lose by selecting a categorical exposure versus a continuous exposure in a generalised linear model (GLM).
I didn't have problems calculating power using ...
- 51
1
vote
2
answers
656
views
Does using robust standard errors change effect size?
I ran some linear regressions in R using lm, with an interaction term (cat x cat) as the predictor of interest (and also incorporating a covariate). To calculate the effect sizes of the interaction as ...
- 23
1
vote
0
answers
29
views
Does the lack of a basic effect preclude more complex post-hoc analyses?
I (Psychology PhD student) have a large dataset consisting of a variety of numerical variables that I predicted may be related to one another. I have performed some basic statistical analyses (a ...
- 11
1
vote
0
answers
18
views
Examples of statistical research affected by errors in package/library code
I can't figure out if this belongs in Cross Validated, Stack Overflow, or Academia, but I think this may be a good place to start:
Can anyone suggest examples of published prospective/retrospective/...
3
votes
1
answer
386
views
What is the effect size `h` in `pwr::pwr.p.test` in `R`?
The R package pwr has a function for calculating the required sample size for a one-sample proportion test (I assume a z-test ...
- 62.4k
1
vote
1
answer
126
views
Can a consensus be reached on the exact meaning of "Cohen's D" and "Hedge's G"?
This question: Difference between Cohen's d and Hedges' g for effect size metrics
shows there are at least two different interpretations each of both Cohen’s D and Hedge’s G, one of them in ...
- 1,212
4
votes
1
answer
696
views
Determining direction of effect in Wilcoxon Test and interpreting pseudomedian in R
I originally posted this on Stack Overflow (https://stackoverflow.com/questions/72649894/determining-direction-of-effect-in-wilcox-test-and-interpreting-pseudomedian), but I realized it fits better ...
- 61
2
votes
0
answers
166
views
Meta-analysis in R: multiple effect sizes with unknown correlation
I'm working on a meta-analysis with multiple effect sizes reported in most studies. My effect sizes are SMD between two groups (health/ disease) and the reason there is multiple effect sizes per study ...
- 21
4
votes
1
answer
184
views
Why I can recover the total effect with OLS but not with logistic regression?
The dataset ISLR2::Default contains one observation per individual.
The variable Default is "Yes" if the individual ...
- 2,261
1
vote
0
answers
18
views
Effect size and statistical significance correspondence
While I understand that it is possible to have a small effect size with statistical significance and a large effect size without statistical significance, is it ever possible to find this across two ...
- 11
3
votes
1
answer
131
views
Building a bootstrapped distribution to test reliability of empirical data
I have data from a study where subjects (considered as "generations" in a Telephone-like game) sang short melodies from one to the next, starting from a randomly-generated melody (seed) ...
- 2,370
1
vote
0
answers
387
views
lmer anova effect sizes
I would like to compute the marginal R squared for the whole factors from an anova table. This post shows how to get the effect size for each fixed effect based on the ...
- 551
2
votes
1
answer
347
views
Spearman's Correlation and effect size
I have completed data collection and I have a Spearman's correlation of .64. My advisor is wanting me to report the effect size and from everything I have read and can find it appears that .64 would ...
- 21
0
votes
0
answers
29
views
Intuitive effect size (and CI) for one-way comparison of group means?
I'm a big fan of common language effect sizes (e.g. McGraw and Wong's CL or Cohen's U3 - "what proportion of group 1 are higher than the average for group 2"). I've been scrabbling around ...
- 775
0
votes
0
answers
118
views
Effect size: difference between Cohen's d and Wilcoxon's r?
In the simple case where I have N trials with mean result $\mu$ and I want a measure of how on the whole they differ from a theoretical model of a normal distribution with mean 0 and standard ...
- 109
0
votes
1
answer
120
views
Conflict Between Confidence Interval and P-values - One Sample T-Tests
I'm conducting one-sample t-test using R. I calculated p value, effect size, and confidence interval, but they seem to be conflicting. Here is a sample code and output:
...
- 189
0
votes
0
answers
176
views
Power analysis: How do you calculate the power (1-B) based on the matched-pairs biserial correlation effect size?
I am running a Wilcoxon-signed rank test, and as suggested by King et al., the best effect size to report for this analysis would be the matched-pairs rank biserial correlation.
How is the power (1-B) ...
- 21
1
vote
1
answer
186
views
How to power a strong superiority hypothesis in an A/B test
Suppose I would like to perform an A/B Test, where it only makes sense to implement the "treatment" over the "control" if the "treatment" is atleast 2% better than the &...
- 341
4
votes
1
answer
2k
views
Small/Medium/Large Cramér's V effect size tables available for degrees of freedom > 5?
I've come across numerous tables published online categorizing Cramér's V effect sizes for categorical tables with degrees of freedom $\leq$ 5, for example:
$$
\begin{array}{c|lcr}
\text{Degrees of ...
- 6,104
1
vote
0
answers
181
views
Integrate chi-square-tests in meta-analysis
I would like to do a meta-analysis, based on community-based intervention studies. In the studies the chi-squares and the p-values are given. But the sample size is related to entire cities, so I don'...
- 11
1
vote
0
answers
63
views
Adjusting effect sizes, p values, and CIs for sequential analysis via simulation
It's fairly easy to write a simulation procedure to calculate, for any given global type 1 error rate, adjusted local "stopping" alphas for interim "looks" in a sequential design, ...
- 234
0
votes
1
answer
461
views
effect size for pairwise permutation
I am reading a paper that is using permutations to compare the means of two different treatment groups (a nutrition study, took minimum - maximum) that have low sample sizes, and so the groups are ...
- 13
0
votes
0
answers
1k
views
Effect size measure for post-hoc tests after nonparametric tests: Alternatives to calculating r from z score?
I've calculated Friedman's tests for my data and want to report an effect size with the individual post-hoc tests that followed on from the statistically significant Friedman's test.
I use the formula ...
- 663
1
vote
0
answers
23
views
common effect sizes in psychological research and calculation/ transformation
I am working on an empirical literature analysis.
For that I have to code some variables in a specific manner (analysis includes 150 published studies in the field of behavior Neuroscience, ...
- 11
0
votes
1
answer
1k
views
Sample size calculation for a three arm clinical trial study
I have a 3 arm clinical study with two drugs and placebo on the three arms respectively. I intend to perform a Log-rank test on it.
I would like to know about the best software which is globally ...
2
votes
1
answer
45
views
Extracting repeated measures standardised difference from studies for meta-analyses (unknown correlation between measures)
I am conducting a meta-analysis and trying to extract standardised differences from a study using pre-post (or repeated measures) design. I am given the Mean, SD, and Sample Size for the pre- and post-...
- 73
2
votes
2
answers
1k
views
Effect size - is correction for multiple comparison needed?
I have a general question on whether effect sizes need to be corrected for multiple comparisons. I need to conduct 8 t-tests, in order to test whether values from 8 different conditions different ...
- 21
1
vote
1
answer
903
views
Calculate power-effect size relationship for difference in difference of proportions
I am trying to conduct power calculations for a "Before-After-Control-Impact" type of study focused on public opinion variables.
Ideally, I want to look at the relationship between effect ...
- 117
1
vote
0
answers
157
views
Using Cohen's f squared to calculate effect size between rank-based correlation coefficients
Is it reasonable to use a Cohen's f squared to calculate an effect size between two Spearman's correlation coefficients that are squared and equivalent to R squared?
The idea is that this comparison ...
1
vote
0
answers
123
views
Should I use effect sizes from multiple regression or raw correlations for meta-analyses?
I am conducting a meta-analysis and have encountered a study that provides both correlation coefficient and multiple regression statistics for the same effect.
Is using effect sizes from multiple ...
- 73
0
votes
0
answers
674
views
Arcsine transformation for two sample proportion test
I'm planning a two sample proportion test and was asked to calculate what sample size is needed to show a certain effect size at a give level of power and significance.
The book I'm using (Statistical ...
- 101
1
vote
0
answers
45
views
Correlation coefficients when oversampling the extremes of a normal distribution
Running some simulations in which I generated two correlated normal variables (rho=0.3), and then took samples of this population in which the extremes of the population where oversampled (...
- 11
1
vote
2
answers
146
views
Is this a known quantity or a random variable?
For two random variables $X_1$, $X_2$, probability of superiority ($PS$) is defined as the probability that a randomly chosen $X_1$ is greater than a randomly chosen $X_2$:
$PS \equiv Pr(X_1 > X_2) ...
- 1,231
1
vote
0
answers
100
views
Formal comparison of two mediation effects
Is there a method to statistically compare two mediation effects, given the same IV and DV? We have two different mediators and we want to compare the resulting models, as in:
Model 1: $X – M_{1} – Y$
...
- 363
0
votes
1
answer
2k
views
Calculating effect sizes for mediations using Process in SPSS
Can someone please help me. Last month I passed my PhD viva (yay!) with minor revisions and one of the revisions is providing effect sizes for me separate studies. For one of my studies I used the ...
- 21
1
vote
0
answers
27
views
Crude difference vs effect size
Let's assume I have two groups of animals with the following weights and the average difference of 8 kilograms:
Could you explain in layman's language, how the calculate difference differs from the ...
- 2,197
1
vote
0
answers
68
views
power vs effect size u-shaped relationship
I'm running a simulation to assess what my minimum detectable effect size for a synthetic control study where the sample size is fixed and I'm noticing a strange relationship between power and effect ...
- 141
2
votes
0
answers
45
views
How do we call the distribution of a ratio consisting of a constant divided by a truncated normal random variable?
I have been thinking for a while about the following problem of Cohen's d effect size measure $d={\frac {\mu _{1}-\mu _{2}}{\sigma}}$ in random-effects meta-analyses for which I cannot find a more ...
- 393
2
votes
1
answer
303
views
Chi-square, effect size and unbalanced groups
For context: I should note in advance I am a relative beginner with this.
Data context
I have data on some 600 000 persons which includes a column of whether these persons took parental leave or not (...
0
votes
1
answer
1k
views
Effect size for Welch's t-test
I ran an independent sample t-test to compare two groups, student sample and Mturk respondents. The homogeniety of variance was not met (variances are not equal and sample sizes of the two groups are ...
- 1
1
vote
0
answers
113
views
Meta-analysis of data with no standard deviation
I am carrying out a Meta-analysis on the impacts of forest logging on Soil. One of the parameter which I want to investigate is the qbs-ar index, which is a index of soil biological quality. This ...
1
vote
1
answer
84
views
Do Odds Ratios (OR) have a similar ranking categorisation as Cohen's conventions?
I'm trying to meaningfully interpret the findings from this meta-analysis which states: "Additional evidence supported the association between fever during pregnancy and increased risk for NDD in ...
- 93
6
votes
1
answer
604
views
How do I summarize the “contribution” of covariates in a GLM?
Qualitatively, I want to answer the question:
Given that the effect of a covariate is statistically significant (as assessed by the p-value / confidence interval of its coefficient), how important is ...
- 335
0
votes
1
answer
43
views
What statistics can I use in a case study to measure pre to post test changes
I am writing a methods section on technical refinement in sport. The study will be on a single participant (Case study) what statistics can I use to measure pre to post-test changes?
I want to see if ...
- 43
1
vote
1
answer
2k
views
Effect Size interpretation for GLM (Logit)
I am using the following code from effectsize package in R:
...
- 35
1
vote
0
answers
27
views
Is Cohen's h appropiate to compare two different F1 scores?
I recently learnt about Cohen's h and its used in power analysis. I wondered if using this metric for comparing two different F1 scores is adequate.
If I have two models with two different F1 scores, ...
- 217