1
$\begingroup$

I am doing a one way analysis of variance to test whether the mean of an observation varies across groups. I'm using R's oneway.test rather than anova because I have thousands of groups and oneway.test is faster. How can I measure the effect size? Is there a way to calculate the $\eta^2$ using the output of oneway.test?

This is how I am doing my test:

oneway.test(o ~ g,
   data=data.frame(o=rep(c(1:100),10)+rnorm(1000,sd=0.1),g=rep(c(1:100),10)),
   var.equal = T) 

I have 8500 groups, and each group has a different number of members, say between 3 and 10. Obviously with these many groups, I expect to get a very significant p-value. That's why I want to find the effect size, because for a large N, even a very small non-zero effect will give a significant pvalue.

$\endgroup$
4
  • $\begingroup$ Please indicate your n-sizes and n for control group and experimental groups. $\endgroup$
    – user10619
    Commented Mar 2, 2017 at 2:51
  • $\begingroup$ @subhashc.davar I just added information about the numbers. $\endgroup$ Commented Mar 2, 2017 at 3:31
  • $\begingroup$ what is p-value and how do you compute? Moreover it is not clear whether your interest in the question is geared to ascertain validity of the given program or anything else - the statistical procedurre to generate a effect size. $\endgroup$
    – user10619
    Commented Mar 2, 2017 at 15:23
  • $\begingroup$ p-value is used in statistics and psychometry. It is used as a test- statistic as well as rho. $\endgroup$
    – user10619
    Commented Mar 2, 2017 at 15:27

1 Answer 1

2
$\begingroup$

In the absence of library function, one can manually calculate the $\eta^2. $Suppose the data is as follows:

set.seed(1); df <- data.frame(o=rep(c(1:10),4)+rnorm(40),g=as.factor(rep(5*c(1:10))))

I am computing the oneway test as

oneway.test(o ~ g, df, var.equal=T)

If I use aov, I can calculate $\eta^2$ as follows using the lsr package:

> lsr::etaSquared(aov(o~g,df))
     eta.sq eta.sq.part
g 0.9338765   0.9338765

Now, I will calculate the $\eta^2$ manually.

m.i = tapply(df$o, df$g, mean)
v.i = tapply(df$o, df$g, var)
n.i = tapply(df$o, df$g, length)

SS_within = sum((n.i-1) * v.i)
SS_between = sum(n.i * (m.i - mean(df$o))^2)
SS_total = (length(df$o)-1) * var(df$o)
SS_between/SS_total
[1] 0.9338765

which is the same as that calculated by the lsr package.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.