Tagged Questions

Effect size is "a measure of the strength of a phenomenon or a sample-based estimate of that quantity" [Wikipedia].

learn more… | top users | synonyms

2
votes
0answers
21 views

How to use delta method for standard errors of marginal effects?

I am interested in better understanding the delta method for approximating the standard errors of the average marginal effects of a regression model that includes an interaction term. I've looked at ...
1
vote
0answers
5 views

Statistical comparison of 2 dependent cohen's ds

How would I statistically test the significane of the difference between two Cohen's d derived from different tests within the same sample? Specifically, in my neuroimaging data metabolism in brain ...
1
vote
1answer
44 views

Effectsize difference between Mann-Whitney U and t-Test

I have data that is officially not normally distributed (Kolmogorov-Smirnov test < .000 BUT N=around 1200 so K-S not really reliable). In plot I can see that form looks good but negatively skewed. ...
4
votes
2answers
64 views

Effect size for a one-sample t-test

Imagine I work in a screw factory (which is not true). We have been producing a series of 5cm-long (on average) screws for years. I want to modify the production parameters in order to save some money ...
0
votes
1answer
21 views

Effect sizes for post-hoc paired comparisons in one-way ANOVA

Is there any alternative (like Cohen's d in t-test) to report about effect sizes for post-hoc paired comparisons in a one-way ANOVA (3 factor levels, correlated samples)?. Many thanks!!
0
votes
1answer
62 views

Interaction terms and effect sizes in multiple regression

I know there are similar questions on here, but I can't quite find an answer that covers all of what I need. I am running multiple regression in r with two predictor variables and sometimes an ...
7
votes
1answer
303 views

Why is usually the acceptable probability of type 1 and type 2 errors different?

This question is raised by my supervisor and I don't know how to explain. Usually the accepted confident level is 0.95 which means that the probability of type 1 error is 5%. But the usually ...
0
votes
0answers
50 views

Effect sizes in within-subject design on R

I'm doing a meta-analysis but I need your precious help on couple things : 1) First, on R, in the compute.es package, I found only effect sizes computation formula ...
1
vote
1answer
36 views

When testing for a difference between effect sizes, is test power affected by their respective magnitude?

When comparing two effect sizes with a statistical test, does it matter for the test's power whether these effect sizes themselves are large or small? Imagine you are planning an experiment in which ...
3
votes
1answer
83 views

How to get proper effect size from LMM?

I now have a problem with effect size of LMM. Someone insisted that I should have effect sizes after p values. I then thought that I can use 'estimate' in the output of pairwise comparison using ...
0
votes
1answer
19 views

How are these negative values understood?

I can't find literature to understand this table. What I know doing my research is that an effect size should be between 0 and 1. when 0.2 is slow, 0,5 medium and 0.8 and higher , high. But In an ...
-1
votes
0answers
24 views

Mann-Whitney U test [closed]

I am comparing 2 separate datasets with the Mann-Whitney U test. I am analyzing three separate parameters present in both datasets and want to know which parameter gives greatest separation between ...
0
votes
0answers
53 views

3 way mixed ANOVA & pairwise comparisons effect size

im in desperate need of some help for my data analysis assignment! I have a 3 way mixed anova- where I have 2 sig main effects, one significant two way interaction and the sig three way ...
1
vote
1answer
62 views

Calculating effect size (partial eta squared) for a planned comparison effect

I have a between-subjects ANCOVA (factors A and B, and covariate C), for which A*B is significant. I decomposed the interaction with a planned comparison described by the contrast A=(-1,1), B=(0,1), ...
1
vote
1answer
69 views

How to perform a meta-analysis of studies without a control group?

I want to meta-analyse some studies which I've already collected and review. However, the studies did not used control groups, due to the types of questions used. How can I do the meta-analysis or ...
2
votes
0answers
58 views

Is there an effect size for Kappa's?

I am staring a project on comparing standard ways of creating a classifier with some heuristic methods. The heuristic methods should result in a faster training for the classifier but should result in ...
0
votes
0answers
19 views

What is a “concordance score” for regression coefficients?

I came across this "concordance score" in a set of slides called Penalized regression methods for ranking variables by effect size, with applications to genetic mapping studies, by Ji Zhu: $$ ...
2
votes
3answers
49 views

Does the order of magnitude of sample values influence p-values and effect size?

When using an independent-samples t-test to compare two variables, do the absolute values of the samples affect the test statistic and the effect size? In other words, is it easier to obtain ...
2
votes
0answers
44 views

Statistically correct to compare two Spearman's $\rho$-Vaules?

I have different variables (A, B, C, D) which are all ordinal scaled. I have another variable (Z) which is ordinal scaled, too. My sample size is approx. 1.500. Is it in the statistical sense ...
6
votes
2answers
194 views

Calculating effect sizes and standard errors for the difference between two standardized mean differences

I have two related questions, both of which are related to a meta-analysis I am performing where where the primary outcomes are expressed in terms of the standardized mean difference. My studies have ...
0
votes
0answers
52 views

Calculating Hedge's g and its variance from t-statistic

I believe there is a way to calculate Hedge's g and its variance from a t-statistic. I have found the formulas to calculate Hedge's g from one-sample t, and from independent groups t. one sample: ...
0
votes
0answers
29 views

Obtaining Cohen's d from z-test statistic

After an ANOVA, I run post-hoc tests with the glht function in the multcomp package. glht ...
4
votes
3answers
90 views

Understanding $\chi^{2}$ and Cramér's $V$ results

I have two groups of buyers, A and B, and I want to test whether the difference between the percentage of them who would buy a ...
0
votes
0answers
39 views

Effect sizes from regression analysis and factor loadings

To do a meta analysis, I try to calculate effect sizes. In an article by Shah and Ward, the authors do a regression analysis after combining various factors by a CPA to four factors. The factor ...
0
votes
0answers
29 views

power analysis when mean and standard deviation are given

I have been asked to perform a sample adequacy test (i.e., I suppose, to verify what is the minimum difference that can be quantified between two grous) where a new sample (of which mu, sigma and ...
1
vote
0answers
21 views

simulate ANOVA data to follow a specific eta-squared

I am simulating RM-ANOVA data with three treatment conditions (one-way within subjects designs). I would like to be able to add an effect to the data so that eta-squared has a specific value. ...
0
votes
1answer
70 views

Question about EFFECT SIZE

I have a question. If I found the effects of a variable is insignificant (p>.05), but the effect size d is 0.2 (small effect). So I used GPower to calculate the power and I found it is 0.14, which ...
3
votes
2answers
48 views

What are real strengths of association when using Pearson correlation?

I have located a table that breaks downs Pearson's correlation values into 3 categories. See below: My question is this. One could have a large positive correlation of 1 as well as a perfect ...
0
votes
0answers
65 views

eta squared and partial eta squared: do they have the same effect size categorization (by cohen)?

I was computing partial eta squared for an ANOVA. I can only find a categorization for eta squared which are either 0.02 (small), 0.13 (medium), 0.26 (large) or .001 (small), .06 (medium) or .12 ...
0
votes
0answers
35 views

after using AIC, how to determine the contribution or effect size of a individual covariate?

I am confused and looking for advise. I have found myself in this same situation repeatedly in the last few months. I want to know if covariate X is influential or important. However, I also ...
0
votes
0answers
15 views

MLR equivalent for repeated measures, multiple correlated responses

I'd like to determine the best approach to determining effect sizes for a set of experimental data with multiple correlated responses. Due to repeated measurements and high correlation between ...
1
vote
1answer
32 views

Using two one variable linear regressions on a single response variable to compare explanatory variables

I ran two single variable linear regressions $A$ and $B$. $A$ had a relatively large effect size $R^2$ = $.68$ while $B$ had $R^2$ = $.10$. Regressing the explanatory variable from $A$ (= $a$) on ...
0
votes
1answer
46 views

Designing Frequentist A/B Experiments

What parameters should one pre-specify before designing/running an experiment? (e.g. a two-sample, one-side, t-test) My understanding is that on establishes: Null hypothesis Maximum Significance ...
3
votes
0answers
38 views

Estimating Sample sizes for Null Hypothesis Testing

Wikipedia has the following table for computing sample sizes:                                       ...
0
votes
0answers
25 views

Moderator Sample Size Discrepancy in Meta Analysis

I'm coding a meta analysis and have come across a paper that seems to have significant variation in the number of participants who answered specific demographic questions. The total sample size is $N ...
1
vote
1answer
43 views

Pre-post treatment design: accounting for reduced effect of treatment in baseline high scorers

I'm planning a study in which I want to test the effect of a treatment on a dependent psychometric variable. I expect subjects who score lower at baseline to benefit more from the treatment (larger ...
8
votes
1answer
136 views

When/why does the central tendency of a resampling simulation markedly differ from the observed value?

Should one always expect the central tendency (i.e., mean and/or median) of a bootstrapped sample to be similar to the observed value? In this particular case I have responses that are distributed ...
2
votes
1answer
41 views

Proportionality of significance and effect size

Is it true that, given a fixed sample size N, it can be said that some measure of effect size and the measure of statitical significance (e.g., 1-p) are proportional? In other words, that, when ...
4
votes
1answer
63 views

Calculating CIs for $\eta^2$ via Z scores - sample size?

In the thread Confidence Interval for $\eta^2$ it was proposed that if only limited statistics are available (in my case, F, df1, df2, means), one could calculate the 95% CI for $\eta^2$ by: ...
0
votes
1answer
31 views

Good Literature on effect sizes

reporting effect sizes is pretty common nowadays, but I couldn't yet find any good book that compares different effect sizes values (d, g, r, etc.). I know the article by Rosnow and Rosenthal (2009), ...
0
votes
0answers
43 views

Effect size for indirect effect with a binary outcome

Is it possible to obtain the effect size of a mediation with a binary outcome? I am looking to report an effect size specifically for the indirect effect. I have already reported the effect sizes of ...
3
votes
1answer
629 views

How to convert from Mann-Whitney U to Z (or other effect size)?

I'm looking at multiple studies. I do not have the original data. One study I'm looking at provides means, standard deviations, and Mann-Whitney $U$ values (with $p$). (How) can I convert from ...
1
vote
2answers
55 views

When are odds ratios worse/better than risk ratios as measures of the effect size of associations between categorical variables?

Wikipedia gives some information on this. However, I seek more detailed information on when odds ratios/risk ratios are better, and why one is better than the other in certain circumstances.
1
vote
1answer
133 views

Does one lose information or precision when calculating effect sizes from statistical values rather than from the raw data?

Let's say that I want to compute the effect size of a certain intervention, for which I have both the raw data and some statistical value that can be transformed to an effect size. For example, if one ...
7
votes
2answers
172 views

In a meta-analysis, how should one handle non-significant studies containing no raw data?

Let's say that I'm conducting a meta-analysis, looking at the performance of group A and group B with respect to a certain construct. Now, some of the studies that I'll come across will report that no ...
2
votes
1answer
107 views

direction of effect size

Should my effect size be positive or negative when the control group gains more weight than the experimental group (when it is hypothesised that the experimental group would lose more weight i.e., the ...
0
votes
0answers
154 views

one-way MANOVA effect size interpretation

I'm having problem understanding effect sizes in one-way MANOVA model. In my case, I'm having a 13 variables which I used for clustering and then the produced ...
7
votes
2answers
329 views

Bayesian analysis of contingency tables: How to describe effect size

I'm working through the examples in Kruschke's Doing Bayesian Data Analysis, specifically the Poisson exponential ANOVA in ch. 22, which he presents as an alternative to frequentist chi-square tests ...
0
votes
0answers
46 views

Variable magnitudes and principal component analysis

I am completing some PCA on various sets of data, some with 30 time-series variables, some with about 100 (meaning matrices with between 30 and 100 columns). I know that the covariance matrix is ...
1
vote
1answer
47 views

(Inverse) proportionality between significance and effect size, for a fixed sample size?

This is a question to hopefully untangle the sometimes confounded notions of statistical significance and effect size. As far as I understand, sample size intermediates the relationship between them, ...