Effect size is "a measure of the strength of a phenomenon or a sample-based estimate of that quantity" [Wikipedia].

learn more… | top users | synonyms

1
vote
1answer
18 views

Calculating effect size (partial eta squared) for a planned comparison effect

I have a between-subjects ANCOVA (factors A and B, and covariate C), for which A*B is significant. I decomposed the interaction with a planned comparison described by the contrast A=(-1,1), B=(0,1), ...
1
vote
1answer
14 views

How to perform a meta-analysis of studies without a control group?

I want to meta-analyse some studies which I've already collected and review. However, the studies did not used control groups, due to the types of questions used. How can I do the meta-analysis or ...
2
votes
0answers
49 views
+50

Is there an effect size for Kappa's?

I am staring a project on comparing standard ways of creating a classifier with some heuristic methods. The heuristic methods should result in a faster training for the classifier but should result in ...
0
votes
0answers
9 views

What is a “concordance score” for regression coefficients?

I came across this "concordance score" in a set of slides called Penalized regression methods for ranking variables by effect size, with applications to genetic mapping studies, by Ji Zhu: $$ ...
2
votes
3answers
28 views

Does the order of magnitude of sample values influence p-values and effect size?

When using an independent-samples t-test to compare two variables, do the absolute values of the samples affect the test statistic and the effect size? In other words, is it easier to obtain ...
2
votes
0answers
38 views

Statistically correct to compare two Spearman's $\rho$-Vaules?

I have different variables (A, B, C, D) which are all ordinal scaled. I have another variable (Z) which is ordinal scaled, too. My sample size is approx. 1.500. Is it in the statistical sense ...
6
votes
1answer
119 views

Calculating effect sizes and standard errors for the difference between two standardized mean differences

I have two related questions, both of which are related to a meta-analysis I am performing where where the primary outcomes are expressed in terms of the standardized mean difference. My studies have ...
0
votes
0answers
21 views

Calculating Hedge's g and its variance from t-statistic

I believe there is a way to calculate Hedge's g and its variance from a t-statistic. I have found the formulas to calculate Hedge's g from one-sample t, and from independent groups t. one sample: ...
0
votes
0answers
9 views

Obtaining Cohen's d from z-test statistic

After an ANOVA, I run post-hoc tests with the glht function in the multcomp package. glht ...
4
votes
3answers
61 views

Understanding $\chi^{2}$ and Cramér's $V$ results

I have two groups of buyers, A and B, and I want to test whether the difference between the percentage of them who would buy a ...
0
votes
0answers
29 views

Effect sizes from regression analysis and factor loadings

To do a meta analysis, I try to calculate effect sizes. In an article by Shah and Ward, the authors do a regression analysis after combining various factors by a CPA to four factors. The factor ...
0
votes
0answers
20 views

power analysis when mean and standard deviation are given

I have been asked to perform a sample adequacy test (i.e., I suppose, to verify what is the minimum difference that can be quantified between two grous) where a new sample (of which mu, sigma and ...
1
vote
0answers
14 views

simulate ANOVA data to follow a specific eta-squared

I am simulating RM-ANOVA data with three treatment conditions (one-way within subjects designs). I would like to be able to add an effect to the data so that eta-squared has a specific value. ...
0
votes
1answer
58 views

Question about EFFECT SIZE

I have a question. If I found the effects of a variable is insignificant (p>.05), but the effect size d is 0.2 (small effect). So I used GPower to calculate the power and I found it is 0.14, which ...
3
votes
2answers
38 views

What are real strengths of association when using Pearson correlation?

I have located a table that breaks downs Pearson's correlation values into 3 categories. See below: My question is this. One could have a large positive correlation of 1 as well as a perfect ...
0
votes
0answers
24 views

eta squared and partial eta squared: do they have the same effect size categorization (by cohen)?

I was computing partial eta squared for an ANOVA. I can only find a categorization for eta squared which are either 0.02 (small), 0.13 (medium), 0.26 (large) or .001 (small), .06 (medium) or .12 ...
0
votes
0answers
15 views

after using AIC, how to determine the contribution or effect size of a individual covariate?

I am confused and looking for advise. I have found myself in this same situation repeatedly in the last few months. I want to know if covariate X is influential or important. However, I also ...
0
votes
0answers
6 views

MLR equivalent for repeated measures, multiple correlated responses

I'd like to determine the best approach to determining effect sizes for a set of experimental data with multiple correlated responses. Due to repeated measurements and high correlation between ...
1
vote
1answer
26 views

Using two one variable linear regressions on a single response variable to compare explanatory variables

I ran two single variable linear regressions $A$ and $B$. $A$ had a relatively large effect size $R^2$ = $.68$ while $B$ had $R^2$ = $.10$. Regressing the explanatory variable from $A$ (= $a$) on ...
0
votes
1answer
41 views

Designing Frequentist A/B Experiments

What parameters should one pre-specify before designing/running an experiment? (e.g. a two-sample, one-side, t-test) My understanding is that on establishes: Null hypothesis Maximum Significance ...
3
votes
0answers
27 views

Estimating Sample sizes for Null Hypothesis Testing

Wikipedia has the following table for computing sample sizes:                                       ...
0
votes
0answers
13 views

Moderator Sample Size Discrepancy in Meta Analysis

I'm coding a meta analysis and have come across a paper that seems to have significant variation in the number of participants who answered specific demographic questions. The total sample size is $N ...
1
vote
1answer
31 views

Pre-post treatment design: accounting for reduced effect of treatment in baseline high scorers

I'm planning a study in which I want to test the effect of a treatment on a dependent psychometric variable. I expect subjects who score lower at baseline to benefit more from the treatment (larger ...
8
votes
1answer
132 views

When/why does the central tendency of a resampling simulation markedly differ from the observed value?

Should one always expect the central tendency (i.e., mean and/or median) of a bootstrapped sample to be similar to the observed value? In this particular case I have responses that are distributed ...
2
votes
1answer
39 views

Proportionality of significance and effect size

Is it true that, given a fixed sample size N, it can be said that some measure of effect size and the measure of statitical significance (e.g., 1-p) are proportional? In other words, that, when ...
3
votes
1answer
44 views

Calculating CIs for $\eta^2$ via Z scores - sample size?

In the thread Confidence Interval for $\eta^2$ it was proposed that if only limited statistics are available (in my case, F, df1, df2, means), one could calculate the 95% CI for $\eta^2$ by: ...
0
votes
1answer
29 views

Good Literature on effect sizes

reporting effect sizes is pretty common nowadays, but I couldn't yet find any good book that compares different effect sizes values (d, g, r, etc.). I know the article by Rosnow and Rosenthal (2009), ...
0
votes
0answers
24 views

Effect size for indirect effect with a binary outcome

Is it possible to obtain the effect size of a mediation with a binary outcome? I am looking to report an effect size specifically for the indirect effect. I have already reported the effect sizes of ...
3
votes
1answer
280 views

How to convert from Mann-Whitney U to Z (or other effect size)?

I'm looking at multiple studies. I do not have the original data. One study I'm looking at provides means, standard deviations, and Mann-Whitney $U$ values (with $p$). (How) can I convert from ...
1
vote
2answers
44 views

When are odds ratios worse/better than risk ratios as measures of the effect size of associations between categorical variables?

Wikipedia gives some information on this. However, I seek more detailed information on when odds ratios/risk ratios are better, and why one is better than the other in certain circumstances.
1
vote
1answer
111 views

Does one lose information or precision when calculating effect sizes from statistical values rather than from the raw data?

Let's say that I want to compute the effect size of a certain intervention, for which I have both the raw data and some statistical value that can be transformed to an effect size. For example, if one ...
7
votes
2answers
114 views

In a meta-analysis, how should one handle non-significant studies containing no raw data?

Let's say that I'm conducting a meta-analysis, looking at the performance of group A and group B with respect to a certain construct. Now, some of the studies that I'll come across will report that no ...
2
votes
1answer
77 views

direction of effect size

Should my effect size be positive or negative when the control group gains more weight than the experimental group (when it is hypothesised that the experimental group would lose more weight i.e., the ...
0
votes
0answers
101 views

one-way MANOVA effect size interpretation

I'm having problem understanding effect sizes in one-way MANOVA model. In my case, I'm having a 13 variables which I used for clustering and then the produced ...
7
votes
2answers
271 views

Bayesian analysis of contingency tables: How to describe effect size

I'm working through the examples in Kruschke's Doing Bayesian Data Analysis, specifically the Poisson exponential ANOVA in ch. 22, which he presents as an alternative to frequentist chi-square tests ...
0
votes
0answers
42 views

Variable magnitudes and principal component analysis

I am completing some PCA on various sets of data, some with 30 time-series variables, some with about 100 (meaning matrices with between 30 and 100 columns). I know that the covariance matrix is ...
1
vote
1answer
36 views

(Inverse) proportionality between significance and effect size, for a fixed sample size?

This is a question to hopefully untangle the sometimes confounded notions of statistical significance and effect size. As far as I understand, sample size intermediates the relationship between them, ...
1
vote
0answers
75 views

Does Greenhouse-Geisser correction influence the effect size estimation, the statistical significance treshold, or both of them?

I discussed recently with one of my colleagues about the Greenhouse-Geisser correction for non-sphericity. I told him first that this correction is very conservative and that it would be more ...
3
votes
1answer
89 views

What do you do after testing if two non-normal distributions differ?

This is a follow up question to "How do I test if two non-normal distributions differ? I have 13 distributions, . According to the previous question, I should do the following. Kruskal-Wallis test ...
1
vote
1answer
85 views

Strength of association test with binary variables

I have a dataset with different purchases for two different items from the same users. So the users purchased the two items at different points in time. I also have 3 different variables: ...
0
votes
0answers
20 views

Very different group sizes - categorical variable

I would like to test for a difference between 3 groups for a binomial variable. There are around 400 in one group, 900 in the other group and 100,000 in the third group. I am wondering if this huge ...
1
vote
0answers
103 views

Absolute effect size and relative effect size in HLM

I am having trouble figuring out what the regression equations for HLM models are with absolute effect size and relative effect size provided in a table. E.g., the outcome variable is reading test ...
13
votes
1answer
579 views

Is a priori power analysis essentially useless?

I attended a meeting of the Society for Personality and Social Psychology last week where I saw a talk by Uri Simonsohn with the premise that using an a priori power analysis to determine sample size ...
3
votes
0answers
82 views

Looking for beginner resources with effect sizes

First post here; first I would like to say that I have no background in stats whatsoever and not so much in math either (enriched high school math, and that's it). I started an MA and we have to do a ...
1
vote
2answers
53 views

Does filtering of data by effect size violate some assumption of P value adjustment methods?

I have pre- and post-treatment continuous data for a large number of variables that I am analyzing for treatment effect. Normally I would obtain the P values and then adjust them for multiple testing ...
5
votes
2answers
223 views

Conflict in confidence intervals for mean difference and confidence interval for Cohen'd effect size

I run an experiment to compare the performance of two algorithms. The design of my experiment is paired comparisons. I'm reporting my results in this way: There were no outliers in mean ...
3
votes
2answers
103 views

What does it mean that the true effect size is not known?

I just read this statement: In neuroscience the true effect size is not known What does this mean? In contrast, what does it mean that where the true effect size is known?
3
votes
0answers
52 views

Cohen's d, but using mean absolute deviation

A common measure of effect size used in the social sciences is to look at the standardized difference in means between two conditions (Cohen's d). That is, take the difference in means between two ...
1
vote
1answer
504 views

How to work out Effect Size for a MANOVA using G*Power

I'm using G*Power to work out how many participants I need. I'll be using a MANOVA, I have two independent variables (Male and Female) and they will be answering 13 different questionnaires so 13 ...
0
votes
0answers
154 views

Effect size in plotLMER.fnc

I'm working with linear mixed models. I used plotlmer.func, in the package lmerConvenienceFunctions to build a graphical ...