Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [effect-size]

Effect size is "a measure of the strength of a phenomenon or a sample-based estimate of that quantity" [Wikipedia].

0
votes
0answers
10 views

Sample size for linear Gamma log link model [on hold]

I need to calculate the sample size for a linear Gamma log link model with factor*factor. I know I need an effect size, and power (1- error type 2) and significance (error type 1) of the test. Please ...
1
vote
0answers
24 views

conducting a meta-analysis on pre-post control design studies that use count data

I would like to conduct a meta-analysis on a set of studies that have measured an outcome over a period prior to intervention and post-intervention for a control group and experimental group. A method ...
1
vote
0answers
23 views

Effect size for mixed ANOVA

I have a mixed anova with repeated measures design: two between-subjects factors, doctor (3 levels) and method (3 levels), and one within-subject factor, time (1w, 3w, 2m, and 6m). 3 doctors would be ...
0
votes
1answer
18 views

Measuring predictive accuracy of an ordinal outcome when the predictor is continuous

I'm predicting a response measured by a 5-point likert scale from a model that produces continuous predictions of the same variable. For example: ...
0
votes
1answer
10 views

What is the relation between the effect size and correlation?

Correlation refers to the degree to which a pair of variables is linearly related. The effect size quantifies some difference between two groups (e.g. the difference between the means of two datasets)....
0
votes
0answers
9 views

Slope as measure of effect size (via standardised correlation coefficient)

Does the situation below correspond to similar or to different effect sizes? The (standardised) Pearson correlation coefficient should only depend on the cumulated distances between data points and ...
1
vote
1answer
28 views

Have I calculated my estimated sample size correctly?

I am conducting a study to answer the following question: In women with type 1 or type 2 diabetes in pregnancy, is there a difference in glycated hemoglobin A1C level between women who have a high ...
3
votes
2answers
49 views

Is it possible to meta-analyze means on different scales?

I have study-level means/SDs reflecting depression symptom severity from multiple single-group studies. I do not have access to participant-level data. The studies all use different measures, and I ...
1
vote
1answer
70 views

Cohens d from Mixed method model - for the interaction term

I am doing a mixed method model: m1 <- lmer(DV ~ IV*Country + (1+IV:Country|Region), data = data) I am using the lme.dscore to get the Cohen's d for the model....
0
votes
0answers
9 views

Convert ratios into a standardized effect like Cohen's d

This is my first time working with log data. I have performed a contrast of the difference between two log means and have back-transformed this to give me a ratio of geometric means. I'd like to ...
0
votes
0answers
5 views

Measuring effects of IV (constant) on DV

I have an independent variable (a bias) that is technically a constant. I want to measure the effects of the IV on the DV. How do I do this? I'm using SPSS. Let's say, participants produced A or B ...
0
votes
0answers
21 views

What test to use to compare two proportions (and calculate effect size and confidence interval)?

There are many related questions here and elsewhere, but I found no satisfying answer. There are plenty of related tests and concepts: Z-test, chi-squared test, G-test, McNemar's test, Fisher's exact,...
0
votes
0answers
7 views

error residual identical for within subject ANOVA and between subject ANOVA with double the sample size

I noticed something interesting while playing around with data. If I conduct a 2-by-2 within subject ANOVA with 20 subjects my sum of square residuals look like this: ...
0
votes
1answer
19 views

Before/After unpaired study with control villages analysed in R

For the implementation of a project, we have been following household access to services in two groups of villages, one an intervention group (I) and one a control group (C). Access as a Yes/No ...
1
vote
1answer
50 views

Calculating effect size for a study with no groups

I am working on a meta analyisis of the effects of e-learning in an organizational context. I have prepared a list of about 25 studies to analyze and I have organized all the path coefficients based ...
1
vote
2answers
35 views

Significance of effect size less then estimated by power

Suppose I estimate a sample size needed for a hypothesis test of proportion difference, using a binomial distributional assumption, with power 80% and effect size 25%. Then I perform an A/B ...
1
vote
0answers
24 views

Meta-analysis for case control studies

I am working on a method of meta-analysis for case-control studies by stratifying cases and controls then calculating the effect size. Can I do a meta-analysis including only case-control studies for ...
1
vote
2answers
20 views

Cohen's d and the use of the standard deviation of the control group

In this paper Cumming, G. (2014). The new statistics: Why and how. Psychological science, 25(1), 7-29. one can read this about Cohens' d: For two independent groups, if we assume homogeneity of ...
0
votes
1answer
19 views

Resource for an explanation of Sample size calculation when $\alpha$, Power and Effect Size for t-Test are given

For a simple z-Test or t-Test (one-sample or two-independent-sample with $n_1 = n_2$ and equal variance) I want to calculate the required sample size $n$ when the significance level $\alpha$, power ...
0
votes
0answers
18 views

How to interpret contradicting path coefficients and f² effect sizes in Structural Equation Models (SEM)?

In the evaluation of my structural equation model with the PLS-SEM approach I have path coefficients between two constructs that are significant (p-value < 0.000) but the effect size f² is small ...
0
votes
0answers
32 views

Standard error for difference between two independent gains

I've calculated an effect size for the difference between two standardized pre-post differences of two independent groups of subjects. That is: $[Mean_{dif_{group~1}}/SD_{dif_{group~1}}]-[Mean_{dif_{...
0
votes
0answers
13 views

How does variation in the effect size affects power?

Effect size The effect size affect the power of a statistical test. We typically summarizes the magnitude of the effect of a variable as a single number (which we call the effect size). To my ...
1
vote
0answers
8 views

Inflation of effect size estimation among significant results

Correction for multiple testing is often performed on the alpha value. However, multiple testing is, I think, likely to inflate also the estimated effect size of the most significant effects found. ...
0
votes
1answer
27 views

Conclusions from combination of P value and Effect size

Limitations of P values are being increasingly highlighted in recent literature (e.g. here and here). To evaluate results of unpaired t-test, following is an approach combining P values and effect ...
1
vote
0answers
11 views

Bayesian method to determine effect size

I have data on heights of 2 groups of persons. Usually Cohen's-d is used to determine effect size. Is there any Bayesian equivalent of Cohen's-d which reflects effect size of the difference between ...
0
votes
0answers
24 views

variance of Cohen's d effect size for two “dependent” samples [duplicate]

The derivation of the sampling variance of Cohen's $d$ effect size for the case of two independent samples is well established (see HERE). However, my question is what is the sampling variance of ...
0
votes
0answers
15 views

Hypothesis testing on two disproportionate samples

I want to conduct hypothesis testing on conversion rate of two disproportionate samples. One contains 20k Data points and other contains 180k datapoints. How can I do a hypothesis testing on such ...
1
vote
0answers
34 views

Interpretation of two odds ratios derived from two different cohen's ds, comparability for meta-analysis?

I have a question regarding the conversion of cohen's d into odds ratio (OR) and the comparability of results from two different situations. I wonder if the ORs derived from the two situations can ...
4
votes
3answers
348 views

Why is my p-value correlated to difference between means in two sample tests?

A colleague has recently made the claim that a large p-value was not more support for the null hypothesis than a low one. Of course, this is also what I learned (uniform distribution under the null ...
0
votes
3answers
32 views

Comparing means in large samples: effect size and p-value

I have a large sample (10000+ per location) of temperature measurements which I would like to compare between 3 locations. I'm checking if there is a significant difference in temperture means between ...
0
votes
0answers
20 views

Meta-analysis with response ratios and log transformed variables

I thought I would ask you something on the computation of Response ratios (Yt/Yc) and what to do when variables are already logged for instance income variables. If you have the answer that would be ...
0
votes
1answer
18 views

Standardized Difference to compare prevalence between 4 groups

In this article, https://www.tandfonline.com/doi/abs/10.1080/03610910902859574 Peter Austin describes how to calculate the standardized difference between two groups, pg 1229 and pg 1230. However am ...
0
votes
1answer
53 views

Can eta squared be used for comparing effect size of a categorical (>2 categories) and continuous variable?

I have a linear model with both continuous and categorical (>2 categories) variables. I am aware of other statistics (e.g., AIC sum of weights and lmg from R package relaimpo) that can be used to ...
0
votes
0answers
30 views

Difference between Wald statistics and effect size in regression models

In some of my regression models (using the rms package), I notice a distinct difference between the result of the Wald statistics (estimated using ...
0
votes
0answers
11 views

Effect size on the difference between group differences (ΔA-B vs ΔA-C) using R

I am trying to calculate effect sizes, but all I can find on the web relates only to the effect size of the difference between two groups. What I am interested in, is the effect size of the difference ...
0
votes
0answers
9 views

Is there a formula to convert t-value from a regression parameter to an effect size (d)?

I've seen formulas to convert independent or dependent samples t-tests into effect sizes, but what about converting the t-value from a regression parameter to an effect size?
0
votes
0answers
58 views

Eta squared effect size calculation for two-way repeated measures ANOVA

I am trying to use my R output for my two-way repeated measures ANOVA to calculate the eta-squared effect sizes. Can anyone help with the code for this, or alternatively how to do it by hand from the ...
0
votes
1answer
36 views

Effect sizes independent of sample size

So I have a small sample size (N=24) but found an extremely large effect size for an effect. A common phrase appearing is 'effect size is independent of sample size,' which I had taken to suggest ...
0
votes
0answers
44 views

Effect size in Negative Binomial Mixed Model

Is there a way to calculate Cohen's d (effect size) equivalent for a coefficient in level 1 (i.e., fixed effect) based on the output from a negative binomial mixed model (...
0
votes
0answers
55 views

Non-Parametric Effect Sizes and Power

As my data was extremely skewed, I have ran 2 non-parametric tests: a Mann-Whitney U and a Wilcoxon Signed Rank test. I have read that using Cohen's D as an effect size for such tests is not ...
0
votes
0answers
43 views

Possible to extend Gelman & Carlin's “Beyond Power Calculations” when significance not obtained?

I have recently been spending time trying to understand the content and extent/limitations of this paper: "Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors" by Gelman ...
0
votes
0answers
30 views

How can you meta-analyse continuous and binary repeated measures outcome data?

Context: the random effects meta-analysis is of studies that measure visits to the doctor (visits), as a continuous variable (by reporting mean visits and respective standard deviations) or as a ...
0
votes
0answers
16 views

Standardised effect size and CI for non-independent longitudinal data in a linear mixed-effect model (LMM)

I would like to calculate standardised effect sizes with confidence intervals for fixed effects in a linear mixed effect model (LMM) for data that have an unbalanced repeated measures longitudinal ...
3
votes
0answers
24 views

Why is information gain rarely used as a reported effect size?

Commonly, in reporting effect sizes in medical articles, papers typically report: risk difference (RD), relative risk (RR) and odds ratio (OR) (and for variable timescale cases and other situations, ...
0
votes
0answers
21 views

Why don't thresholds of effect sizes between $r$ and $R^2$ or $f$ and $f^2$ directly correspond?

I recently compared the "rule of thumb" thresholds (Cohen, 1988) of r and $R^2$ f and $f^2$ In a table: The squared versions don't seem to correspond to $r$ and $f$ at all. E.g. a medium effect is $...
4
votes
2answers
52 views

What is the variance of the standardized mean difference of two independent studies?

I have computed the standardized mean differences for a pair of independent samples. But when I Googled for the "variance of the standardized mean difference" I got rather confusing results, with ...
1
vote
2answers
31 views

correlation-based effect size

how can a correlation coefficient of .40 be a "medium" effect size when r2 shows that the variables only have 16% common variance? Seems like the 16% suggests a small effect size. Jim
1
vote
0answers
59 views

Can the r-scale value (for Bayes Factor) be directly based on Cohen's d?

I want to set an r-scale value for a Bayesian t-test (i.e. to calculate Bayes Factor likelihood ratio) based on previous results (i.e., from a posterior). But I simply cannot find a straightforward ...
0
votes
1answer
29 views

How to interpret a statistically signficant finding that becomes unsignificant after bonferroni correlation

I hypothesised I would find a main effect of X which I did (p=0.028), however, upon using the bonferroni correction my alpha dropped to 0.0125, making this result no longer significant. I found a ...
0
votes
1answer
84 views

How can I calculate confidence intervals for epsilon-squared (Effect size for Kruskal-Wallis test) in R? [closed]

I'd like to calculate the confidence intervals for the epsilon-squared effect size statistic in R. Here is the code I'm using to calculate the epsilon-squared value: ...