Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [effect-size]

Effect size is "a measure of the strength of a phenomenon or a sample-based estimate of that quantity" [Wikipedia].

0
votes
0answers
10 views

variance of Cohen's d effect size for two “dependent” samples

The derivation of the sampling variance of Cohen's $d$ effect size for the case of two independent samples is well established (see HERE). However, my question is what is the sampling variance of ...
0
votes
0answers
13 views

Hypothesis testing on two disproportionate samples

I want to conduct hypothesis testing on conversion rate of two disproportionate samples. One contains 20k Data points and other contains 180k datapoints. How can I do a hypothesis testing on such ...
1
vote
0answers
26 views

Interpretation of two odds ratios derived from two different cohen's ds, comparability for meta-analysis?

I have a question regarding the conversion of cohen's d into odds ratio (OR) and the comparability of results from two different situations. I wonder if the ORs derived from the two situations can ...
4
votes
3answers
342 views

Why is my p-value correlated to difference between means in two sample tests?

A colleague has recently made the claim that a large p-value was not more support for the null hypothesis than a low one. Of course, this is also what I learned (uniform distribution under the null ...
0
votes
0answers
20 views

Estimating effect of covariates operating thru another covariate with a twist [closed]

A few things first 1) my title is a poor description of my problem; 2) my description (and notation) is poor; 3) and I'm still trying to figure out how to describe my problem. Say I have data ...
0
votes
3answers
25 views

Comparing means in large samples: effect size and p-value

I have a large sample (10000+ per location) of temperature measurements which I would like to compare between 3 locations. I'm checking if there is a significant difference in temperture means between ...
0
votes
0answers
18 views

Meta-analysis with response ratios and log transformed variables

I thought I would ask you something on the computation of Response ratios (Yt/Yc) and what to do when variables are already logged for instance income variables. If you have the answer that would be ...
0
votes
1answer
16 views

Standardized Difference to compare prevalence between 4 groups

In this article, https://www.tandfonline.com/doi/abs/10.1080/03610910902859574 Peter Austin describes how to calculate the standardized difference between two groups, pg 1229 and pg 1230. However am ...
0
votes
1answer
25 views

Can eta squared be used for comparing effect size of a categorical (>2 categories) and continuous variable?

I have a linear model with both continuous and categorical (>2 categories) variables. I am aware of other statistics (e.g., AIC sum of weights and lmg from R package relaimpo) that can be used to ...
0
votes
0answers
25 views

Difference between Wald statistics and effect size in regression models

In some of my regression models (using the rms package), I notice a distinct difference between the result of the Wald statistics (estimated using ...
0
votes
0answers
11 views

Effect size on the difference between group differences (ΔA-B vs ΔA-C) using R

I am trying to calculate effect sizes, but all I can find on the web relates only to the effect size of the difference between two groups. What I am interested in, is the effect size of the difference ...
0
votes
0answers
9 views

Is there a formula to convert t-value from a regression parameter to an effect size (d)?

I've seen formulas to convert independent or dependent samples t-tests into effect sizes, but what about converting the t-value from a regression parameter to an effect size?
0
votes
0answers
36 views

Eta squared effect size calculation for two-way repeated measures ANOVA

I am trying to use my R output for my two-way repeated measures ANOVA to calculate the eta-squared effect sizes. Can anyone help with the code for this, or alternatively how to do it by hand from the ...
0
votes
1answer
23 views

Effect sizes independent of sample size

So I have a small sample size (N=24) but found an extremely large effect size for an effect. A common phrase appearing is 'effect size is independent of sample size,' which I had taken to suggest ...
0
votes
0answers
32 views

Effect size in Negative Binomial Mixed Model

Is there a way to calculate Cohen's d (effect size) equivalent for a coefficient in level 1 (i.e., fixed effect) based on the output from a negative binomial mixed model (...
0
votes
0answers
25 views

Non-Parametric Effect Sizes and Power

As my data was extremely skewed, I have ran 2 non-parametric tests: a Mann-Whitney U and a Wilcoxon Signed Rank test. I have read that using Cohen's D as an effect size for such tests is not ...
0
votes
0answers
41 views

Possible to extend Gelman & Carlin's “Beyond Power Calculations” when significance not obtained?

I have recently been spending time trying to understand the content and extent/limitations of this paper: "Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors" by Gelman ...
0
votes
0answers
23 views

How can you meta-analyse continuous and binary repeated measures outcome data?

Context: the random effects meta-analysis is of studies that measure visits to the doctor (visits), as a continuous variable (by reporting mean visits and respective standard deviations) or as a ...
0
votes
0answers
14 views

Standardised effect size and CI for non-independent longitudinal data in a linear mixed-effect model (LMM)

I would like to calculate standardised effect sizes with confidence intervals for fixed effects in a linear mixed effect model (LMM) for data that have an unbalanced repeated measures longitudinal ...
3
votes
0answers
21 views

Why is information gain rarely used as a reported effect size?

Commonly, in reporting effect sizes in medical articles, papers typically report: risk difference (RD), relative risk (RR) and odds ratio (OR) (and for variable timescale cases and other situations, ...
0
votes
0answers
20 views

Why don't thresholds of effect sizes between $r$ and $R^2$ or $f$ and $f^2$ directly correspond?

I recently compared the "rule of thumb" thresholds (Cohen, 1988) of r and $R^2$ f and $f^2$ In a table: The squared versions don't seem to correspond to $r$ and $f$ at all. E.g. a medium effect is $...
5
votes
2answers
39 views

What is the variance of the standardized mean difference of two independent studies?

I have computed the standardized mean differences for a pair of independent samples. But when I Googled for the "variance of the standardized mean difference" I got rather confusing results, with ...
1
vote
2answers
23 views

correlation-based effect size

how can a correlation coefficient of .40 be a "medium" effect size when r2 shows that the variables only have 16% common variance? Seems like the 16% suggests a small effect size. Jim
1
vote
0answers
50 views

Can the r-scale value (for Bayes Factor) be directly based on Cohen's d?

I want to set an r-scale value for a Bayesian t-test (i.e. to calculate Bayes Factor likelihood ratio) based on previous results (i.e., from a posterior). But I simply cannot find a straightforward ...
0
votes
1answer
28 views

How to interpret a statistically signficant finding that becomes unsignificant after bonferroni correlation

I hypothesised I would find a main effect of X which I did (p=0.028), however, upon using the bonferroni correction my alpha dropped to 0.0125, making this result no longer significant. I found a ...
0
votes
1answer
57 views

How can I calculate confidence intervals for epsilon-squared (Effect size for Kruskal-Wallis test) in R? [closed]

I'd like to calculate the confidence intervals for the epsilon-squared effect size statistic in R. Here is the code I'm using to calculate the epsilon-squared value: ...
1
vote
0answers
17 views

Choosing the measure of effect size to extract&summarise in a meta-analysis

I am preparing a meta-analysis for a certain therapeutic intervention, and will be interested to quantify/summarise the change in various scores, pre-to-post intervention. Which measure of effect ...
0
votes
0answers
47 views

Applying Cliff's delta to paired data

Wikipedia provides a few ways of calculating Cliff's delta - an effect size quantifying distributional shift. However it only talks about having 2 samples that are unpaired. While one can apply the ...
1
vote
1answer
124 views

Converting between correlation and effect size (Cohen's d)

Several sources (here here here) claim that there is a relation between Cohen's d and Pearson's r if the data is paired (bivariate). This strikes me as odd since, for example, evaluating a "before and ...
1
vote
0answers
15 views

Obtaining measures of effect for contingency tables with multiply imputed data

The epi.2by2 function in the epiR package computes a chi-square test and provides measures of effect when count data are ...
0
votes
0answers
5 views

Are effect sizes calculated for prospective/ retrospective cohort studies that look at odds ratios?

Are effect sizes calculated for prospective/ retrospective cohort studies that look at odds ratios? I apologise if this is a silly question
0
votes
1answer
16 views

Reporting effect aize

Could someone please explain the difference between standard mean difference and adjusted mean difference. I want a common rubric for reporting effect size.
0
votes
0answers
142 views

Interpretation of partial eta-squared in factorial designs

I'm having troubles interpreting and comparing effect sizes in factorial designs based on partial eta squared values. E.g. in a 3x3-within-subjects design with: η²part, A = .2 η²part, B = .4 η²part, ...
1
vote
2answers
81 views

How to determine sample size for Chi-squared test?

Similar questions have been asked here on CV, however I have not found a solution to my specific question yet. I am trying to plan a sample size for chi-squared test. However, I struggle to find the ...
2
votes
2answers
85 views

How to calculate the effect size for a t-test?

The pwr package includes the function pwr.t.test, which relates the following four quantities: sample size effect size ...
1
vote
1answer
17 views

How do I get a measure of effect size for each term in a multilevel model?

For a multilevel model of the form, say: $$Y_{ij} = B_0 + B_1X_{1_{ij}} + B_2(X_1)^2_{ij} + B_3X_{3_{ij}} + B_4X_{1_{ij}}X_{3_{ij}} + B_5X_{5_j} + (u_0 + u_{1_j} + e_{0_{ij}} + e_{1_{ij}})$$ Does ...
1
vote
0answers
16 views

Magnitude of the effects using confidence interval [closed]

I have an assignment (doing regression) and one of the tasks is "Check the magnitude of the effects using confidence intervals.". How can I check this and how to report it in a paper? Using SPSS.
0
votes
0answers
27 views

why is the effect size in sjstats anova_stats NA

I am trying to find the effect size of my mixed effects model. I found sjstats provides several outputs including eta-sq, ...
0
votes
0answers
17 views

How am I able to compute the effect size and variance for the given data?

I am currently studying a research paper in which the following data is given: Control group prevalence estimate: 5.1% (standard error = 1.9), n=138 Treatment group prevalence estimate: 13.0% (...
1
vote
1answer
43 views

Is there a commonly accepted effect size parameter for pairs of Bernoulli processes?

Let $f(x|\rho)$ be the Bernoulli pmf with probability $\rho$ of success. \begin{align} f(x|\rho) = \left\{ \begin{array}{ll} \rho & x = 1 \\ 1-\rho & x=0 \end{array} \right....
0
votes
1answer
19 views

Is it possible to determine significance with a defined alpha (.05) from an effect size estimate with a 95% CI?

Say for example, there is an observed estimated standardized mean difference of $d = 0.44$ with $CI_{95} = [-.10, .89]$. This is the only information we have available (i.e. we do not know the sample ...
0
votes
0answers
20 views

Calculate correlation from paired t test's results

I have the degrees of freedom, difference between means, 95% confidence interval, and p-value of a paired t-test (i.e., a t-test run to analyze pre-post scores). I need to calculate the corresponding ...
0
votes
0answers
45 views

Multiple Regression Effect Size, Significance, and Cohens f^2

I have a multiple regression with a continuous dependent variable, 1 continuous independent variable, and a handful of binary independent variables. The R summary is pasted below: ...
0
votes
1answer
38 views

Which effect size when sample size and variance are not equal

I don't have extensive knowledge in statistics, therefore I have some trouble to understand which metrics I should use for my analysis. I have measure a feature (concentration) of multiple substances ...
1
vote
0answers
21 views

The curious case of the mysterious repeated measures effect size estimate

I am currently in the middle of a stone-cold statisical whodunnit. I need to find the name of an effect size statistic a recently departed colleague used for a project, measuring effect size of change ...
0
votes
1answer
177 views

Relation between t-value and correlation coefficient r

Page 6 of this article shows that following holds for an "independent samples t-test". I was wondering how the equation to the Right of equal sign would change if we consider a "paired samples t-...
3
votes
1answer
53 views

Meta-analysis with response ratio effect size

I am interested in comparing the performance of two different techniques (i.e., Method A vs. Method B) on a continuous outcome of interest. The performance of such techniques is measured through test ...
0
votes
0answers
13 views

Non-parametric equivalent to Z-factor?

It seems that all the ways of calculating standardized effect sizes assume a normal distribution: Z-factor, Cohen's d, SSMD, ... I wonder if there is a non-parametric equivalent to these. In ...
0
votes
0answers
107 views

Converting Cohen's d to r for two paired groups

I know that there is an exact formula for converting a Cohen's d effect size to a Pearson correlation coefficient ($r$) in the context of two 'independent' groups as discussed HERE (see p. 6): $r = ...
0
votes
0answers
26 views

Calculating effect size for EEG data

Our data is from two scalp EEG channels (C3 and C4), which correspond to the sensorimotor cortex on each side of the brain. We are looking at how effect size of a waveform metric differs between ...