I am analysing data which I generated using a simulation model. Due to the stochasticity of the model, the output is also stochastic. I have several dependent variables (let's say X and Y) and vary six different variables (A, B, C, D, E, F) in a full factorial design. Each variable has between 2 and 5 levels.

Three of the variables (A, B, C) are used to simulate different "framework conditions". I am not directly interested in the effect of these variables on my dependent variable, but I'd like to know if there is an effect of the other three variables (D, E, F) on my dependent variable given a set of values for A, B and C. I am not directly interested in the effects of A, B or C.

So far, I tried to use factorial ANOVA, but integrating all the variables A, B, C, D, E and F highly violates the assumptions of (normally distributed residuals, homoscedasticity) as tested by a Shapiro-Wilk- and Levene's-Test. This can easily explained by the fact that the extent of variation and the distribution of the subgroups heavily depends on the demand profile (e.g. small standard deviation for the subgroups if factor C has a specific value).

If I group the dataset by the factors A, B, C and apply individual factorial ANOVAs for the remaining variables (D, E and F), assumptions are not violated.

My question is:

  1. Can I run different ANOVAs for the mentioned subgroups and meaningfully interpret the results?

  2. Do I have to account for multiple testing?

  • $\begingroup$ Maybe you can share (a link to) the data? $\endgroup$ – kjetil b halvorsen Mar 11 at 17:40

Should you account for multiple testing? I am sure opinions will differ, and it will depend on the goals for the analysis, and the number of separate anova's (you didn't tell.) It is probably better to go for one common model. For that, formulate the anova as a regression model, and choose some technique for handling heteroskedasticity in regression models. There are many possibilities, and you did not tell us much context:

To get some more specific advice, maybe you could augment your Q with some plots and model output?

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.