5
$\begingroup$

Everywhere I have read that if you want to test the hypothesis of dependence of a binary outcome on your data, you should not apply ANOVA (if its incorrect, please let me know). We can apply logistic regression, chi square testing etc.

I have come across a paper in the reputed Journal of Vision, where I believe the ANOVA is being misused. I would like to mention that this is not just one case and I have seen many papers like this: Paper

In this paper the authors are mainly seeing how inversion of face images leads to accuracy changes. However, they have written this statement:

ANOVA was conducted with the main factors of Orientation (upright or inverted), Change Type (configural or featural) and Region (eyes or mouth). The results (Figure 3) showed a significant main effect of Orientation, F(1, 21) = 24.35, p < 0.01, and Region, F(1, 21) = 16.97, p < 0.01.`

Can someone help me out by figuring how ANOVA is being applied here? All the 3 parameters Orientation, Change Type and Region are binary. However the authors are reporting the p values and hypothesis significance. Is this the correct way to do so?

I am mainly asking this not just from the viewpoint of this paper. As I have mentioned many authors just take factors that are binary, apply ANOVA and show that one of them is better in a specific task.

$\endgroup$
5
  • $\begingroup$ There's an edit that's just come through - you might want to check that you agree with the change in title (you can change it back if need be). $\endgroup$
    – Glen_b
    Commented Feb 24, 2013 at 23:58
  • 1
    $\begingroup$ @jonsca Oh, I agree with the edit, I was just concerned the OP, being a new user, mightn't realize that if they wanted it changed back, it could be done. $\endgroup$
    – Glen_b
    Commented Feb 25, 2013 at 0:12
  • $\begingroup$ @Glen_b Gotcha. A good call. $\endgroup$
    – jonsca
    Commented Feb 25, 2013 at 0:13
  • $\begingroup$ I think its right that ANOVA should not be used to analyze binary dependent variable. Many people use this to compare means of the binary response variable (0 1) but it should not be used because this seriously violates the Normality and Equal variance assumption. Chi-Square tests are best for these situations. Please let me know if I am wrong. $\endgroup$ Commented Aug 21, 2014 at 9:19
  • $\begingroup$ Sometimes, ANOVA can be used, but together with the variance-stabilizing transform for thwe binomial distribution, which is $\arcsin\sqrt(p)$. $\endgroup$ Commented Aug 21, 2014 at 11:55

2 Answers 2

5
$\begingroup$

You mix up dependent and independent variable. Whereas it is true, that the dependent variable in an ANOVA should not be binary, but interval scaled (i.e., continuous), this is not true for the independent variable. The independent variables are usually categorical (this includes binary variables).

Remember, independent variables are your factors or experimental manipulations, dependent variable is what you measure.

So, what the authors do is totally legitimate, as their dependent variable seems to be some kind of eye movement measure.

$\endgroup$
1
$\begingroup$

Those three factors you mention are IVs (predictors) not DVs (responses).

The DV "Change detection accuracy" (e.g. see fig 3) looks like a count variable (a sum of binary variables, if you like), will be both discrete and heteroskedastic, and may be skew as well - and strictly should probably be handled as a GLM. If the samples are large and the proportions don't vary too much it may not be a very serious problem.

The DV "Response time" (e.g. see fig 4) is continuous, but is likely to be highly skewed and heteroskedastic (so again, likely not well suited for normal theory ANOVA). Speeds (inverse of times) are often much more reasonable on normal assumptions (but even there you can get some degree of skewness or heteroskedasticity). How much of an issue the time variable is it's impossible to judge without looking at the data.

The DV "Number of saccades" is another count variable, and the earlier comments apply

The DV "Saccade distance" may be right skew and mildly heteroskedastic, but probably not as badly as the time variable.

"First fixation" looks like another count variable, ...

.... and so on.

Some of these might be okay handled via ANOVA, I can't tell without the data.

Effect sizes and so on will be fine; the significance levels are what you have to worry about.


rolando2 said:

Homo- or heteroskedasticity is a property of a solution (some would say of a "model"), not of a single variable.

I'm bringing this up here because I want to discuss it in some depth.

Heteroskedasticity is not a property of a single variable, but we're not dealing with a single variable.

We're discussing ANOVA: the existence of a model is already a given - in the sense that we're modeling relations among means of different groups - we have both y and x-variables (the x's will be factors in ANOVA of course).

And that's enough variables to have heteroskedasticity, if it's present.

Since count variables seem to nearly always show variability that relates to the mean in some way (in particular, among groups with very large fitted means compared to other groups, we also see larger variation around the mean), it's important to bring it up.

That this happens is to be expected, since counts are bounded. They cannot go below 0.

Consider a simple one-way ANOVA with (for clarity) multiple count observations per group. Moving across groups, as the mean gets closer to 0, the variability of points about the mean must also tend to be smaller - they are less and less able to go far below the current mean (because of the bound) and so by necessity must not go too far above it (or the mean would tend to be larger than it is).

As a result, when there are count dependent variables, you nearly always find variance that's somewhat related to the mean in the data (not necessarily linearly) -- as long as there's variation in the mean among the groups, we would expect there to be accompanying changes in variances, simply because of the basic (and obvious) case that counts are bounded below.

Which is to say, in the situation in the question, if there's to be variation in the mean (& that expectation is why ANOVA was considered in the first place), then we have every ingredient needed for us to expect heteroskedasticity to be present - if the means differ between the groups, we should typically expect there to be changes in variance.

$\endgroup$
2
  • $\begingroup$ Homo- or heteroskedasticity is a property of a solution (some would say of a "model"), not of a single variable. $\endgroup$
    – rolando2
    Commented Jul 24, 2014 at 11:33
  • $\begingroup$ @rolando2 I wasn't suggesting it was a property of a single variable (though I guess I can see how you might have taken it that way). I've addressed your comments in my answer above. I'll happily expand on them at greater length. $\endgroup$
    – Glen_b
    Commented Jul 24, 2014 at 22:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.