1
$\begingroup$

I'm wondering if anyone can help me with a really tricky one-way independent ANOVA problem I'm having? Here's a bit of background:

There are 3 diagnostic groups (depression, anxiety, non-clinical). The groups are independent, but participants in the depression group might also have an anxiety diagnosis. The dependent variable is a continuous measures of self-esteem. A-priori power analysis indicated that I need around 50 participants per group to detect a medium effect, which I do. However, after doing an initial ANOVA to look at differences between these groups, I need to do a follow-up to see whether controlling for anxiety in the depression group affects the group differences (as they might just be similar on the DV due to shared high levels of anxiety).

I can't use the continuous measure of anxiety I've used to categorise participants into the diagnostic groups as a covariate, as this would remove the effect of anxiety in the anxiety group (which doesn't make much sense!) The only alternative option I've found is to split the depression group into 'depression with anxiety' and 'depression without anxiety' and then run another one-way ANOVA this way. However, the 'depression without anxiety' group is far smaller than 50, and so the test wouldn't be adequately powered.

Can anyone think of a way around this that results in a suitable, adequately powered test? Essentially, I need to control for a variable in one group but not the others!

$\endgroup$
2
  • 2
    $\begingroup$ Welcome to CV. Since you’re new here, you may want to take our tour, which has information for new users. Would you consider editing the title to make it more informative? $\endgroup$
    – T.E.G.
    Sep 9, 2020 at 13:59
  • $\begingroup$ I took the liberty of replacing the title with one that is more informative. $\endgroup$
    – EdM
    Sep 9, 2020 at 18:11

2 Answers 2

2
$\begingroup$

As you seem to have a continuous measure of anxiety, you should take advantage of it.

Instead of continuing with an ANOVA-type analysis you could consider a linear model that specifically allows for both anxiety and depression scores for each individual. If you have anxiety and depression scores for each individual (including the non-clinical group) then you could use those scores plus an interaction between anxiety and depression scores to try to characterize your outcome. If we call the anxiety score "A", the depression score "D", and use ":" to represent the interaction then you could use a linear regression model:

outcome ~ A + D + A:D

That would also cover the non-clinical group, whose members presumably have low "A" and "D" scores. As your "A" score is continuous, this could work quite well even if "D" is binary. That would allow modeling of any combination of anxiety and depression score. This linear modeling of "A" and "D" and their interaction only uses up 3 degrees of freedom, versus the 2 in your ANOVA model, so your 150 cases should still be enough even if you, say, have to do some non-linear modeling of the "A" scores with a spline.

You will, however, have to be careful when interpreting the individual "A" and "D" coefficients with an interaction in the model. The coefficient for "A" will be the association of outcome with "A" when "D" is at its reference value (or 0 if continuous), and that for "D" will be the association of outcome with "D" when your continuous measure "A" is at a value of 0. If the interaction is significant then there won't be a single value representing either of anxiety or depression. That might be more complicated than you were hoping for, but if that's the reality then that is what your audience deserves to know.

That said, I am concerned about the issues raised by @BruceET with respect to re-design of the analysis after the data have been collected. I think this type of analysis will help get around those issues, but I haven't thought fully through whether the procedure for selecting the individuals for this study might lead to some problems if you proceed this way.

$\endgroup$
1
$\begingroup$

Re-designing an experiment after you see the data can be a direct path to false discovery.

If you can use pre-experiment information as criteria to split the Depression group for a post hoc analysis whether the two parts of it differ significantly. If so, then your report needs to admit the difficulty and unorthodox re-design along with findings.

Here are data for a design such as yours.

set.seed(2020)
NC = rnorm(50, 100, 20)
D  = rnorm(50, 115, 30)
A  = rnorm(50, 110, 20)
x = c(NC, D, A)
g = rep(1:3, each=50)
boxplot(x ~ g, col="skyblue2", pch=20)

enter image description here

oneway.test(x ~ g)

        One-way analysis of means (not assuming equal variances)

data:  x and gp
F = 3.5155, num df = 2.000, denom df = 95.537, p-value = 0.03364

A "re-design" based on illegitimate use of data, can produce bogus differences. This is an extreme case, but it is why I mentioned that splitting the Depression group needs to be based on pre-experiment information.

D1 = sort(D)[1:20]    # smallest into D1
D2 = sort(D)[21:50]   # largest into D2
y = c(NC, D1, D2, A)
gp = rep(1:4, c(50,20,30,50))
boxplot(y~gp, col="skyblue2", pch=20, varwidth=T)

enter image description here

oneway.test(y~gp)

        One-way analysis of means (not assuming equal variances)

data:  y and gp
F = 26.23, num df = 3.000, denom df = 65.733, p-value = 2.843e-11

t.test(D1,D2)$p.val
[1] 1.492356e-11
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.