Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I would like to compute the p-value of my two-way ANOVA. The score that I am using to detect significant samples is the eta score which is computed as SS(between)/SS(total).

In many sites I saw that F=Var(between)/Var(within). I want to know if I can consider the eta score as the F-statistic value, and then compute the p-value?

What if I want to compute the p-value for other scores such as eta-partial, or omega?
Is it meaningful if I calculate the p-value for them? Or is it only meaningful for the ratio Var(between)/Var(within)? What difference does it make if I use SS(between)/SS(within) instead of Var?

The formula that I am using to calculate p-value is:

pvalue=-log10(betai(0.5 * df2, 0.5* df1, df2 / (df2 + df1 * eta)));  

based on the book Numerical Recipes in C, where df1 and df2 are degrees of freedom.

Thanks for your help.

share|improve this question
    
@chl thanks for edits. Quite useful. –  Pegah Apr 28 '11 at 18:43
    
It looks like you're confused about F-test for testing two variances (in the NRC++ 3rd ed., I read prob = 2.0*beta.betai(0.5*df2,0.5*df1,df2/(df2+df1*f)), where f=var1/var2, in the section entitled "14.2.2 F-Test for Significantly Different Variances") vs. one- or two-way ANOVA. What you described suggests you're interested in a one-way model (you have one response variable and one explanatory variable, or factor) and want to conduct a statistical test about the effect of the factor with an F-test while providing a measure of effect-size with the eta-squared. Could you confirm? –  chl Apr 28 '11 at 19:02
    
@chl thanks for the answer. I am using 2way-anova, and I have 2 factors ( I did not get what you mean by response variable). I am interested in the variance of one of the factors in comparison to the total variance (both factors) which forms the eta squared. –  Pegah Apr 29 '11 at 8:19
add comment

1 Answer

up vote 2 down vote accepted

The eta-square ($\eta^2$) value you are describing is intended to be used as a measure of effect size in the observed data (i.e., your sample), as it amounts to quantify how much of the total variance can be explained by the factor considered in the analysis (that is what you wrote in fact, BSS/TSS). With more than one factor, you can also compute partial $\eta^2$ that reflect the percentage of variance explained by one factor when holding constant the remaining ones.

The F-ratio (BSS/WSS) is the right test statistic to use if you want to test the null hypothesis ($H_0$) that there is no effect of your factor (all group means are equal), that is your factor of interest doesn't account for a sufficient amount of variance compared to the residual (unexplained) variance. In other words, we test whether the added explained variance (BSS=TSS-RSS) is large enough to be considered as a "significant quantity". The distribution of the ratio of these two sum of squares (scaled by their corresponding degrees of freedom--this answers one of your question, about why we don't use directly SSs), which individually follow a $\chi^2$ distribution, is known as the Fisher-Snedecor distribution.

I don't know which software you are using, but

  • If you have R, everything you need for basic modeling is given in the aov() base function ($\eta^2$ might be computed with etasq from the heplots package; and there's a lot more to see for diagnostics and plotting in other packages).
  • If you're more versed into C programming, you may have a look at the apophenia library which features a nice set of statistical functions with bindings for MySQL and Python.
share|improve this answer
    
great answer. I got the concept. thank you so much. –  Pegah May 24 '11 at 19:08
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.