# Factorial experiment design with incompatible levels in IVs

Is it safe to run a Factorial ANOVA if you cannot collect data for some cells?

Scenario: There are two independent variables, and one dependent variable.

All independent variables are nominal:

• D (data structure) has 4 levels: $d_1$ to $d_4$.

• F (distance function) has 6 levels: $f_1$ to $f_6$.

The dependent variable is interval: Y (outcome) has domain $[0, 1] \subset \mathbb{R}$.

Data looks like this:

(subject, D, F, Y) Example: (user001, $d_1$, $f_1$, 0.3).

Goal: To determine the effect of D and F on Y.

Problem: there is incompatibility between some levels of D and F.

Example: $d_1$ is incompatible with $f_6$. This means that Y cannot be measure in this cell.

There are other incompatibilities:

• $d_2$ is incompatible with $f_6$.

• $d_3$ is incompatible with $f_3$ to $f_6$.

• $d_4$ is incompatible with $f_1$ to $f_5$. This means that two objects represented as data structures of type $d_4$ can only be compared by using the function $f_6$.

There is no problem doing this ... although it will depend on your goals. Eliminating from the design the impossible combinations, you will get a non-regular fraction, but that in itself is not a problem. Main effects can still be estimated. If you are interested in interactions, interaction parameters corresponding to the impossible combinations cannot be estimated, but as they are not meaningful that is not a problem.

You can investigate doing some simulations in R:

D <- factor(paste0("D", 1:4, sep=""))
F <- factor(paste0("F", 1:6, sep=""))
dat0 <- expand.grid(D=D, F=F)
incompat <- c(21, 22, 4, 8, 12, 16, 20)
dat <- dat0[-incompat, ]

N <- NROW(dat)
# Simulate some data:
Y <- rnorm(N, 10, 2)
mod <- lm(Y ~ D + F, data=dat) # Main effects model

summary(mod)

Call:
lm(formula = Y ~ D + F, data = dat)

Residuals:
Min      1Q  Median      3Q     Max
-2.4055 -1.0877  0.2836  0.9961  1.8258

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)  9.71458    1.29380   7.509 6.87e-05 ***
DD2          0.08264    1.19783   0.069   0.9467
DD3          1.37563    1.19783   1.148   0.2840
DD4          5.46891    2.93407   1.864   0.0993 .
FF2          0.90613    1.54639   0.586   0.5741
FF3          0.29150    1.54639   0.189   0.8552
FF4          0.42037    1.54639   0.272   0.7926
FF5         -2.00301    1.54639  -1.295   0.2313
FF6         -2.22783    2.29367  -0.971   0.3598
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.894 on 8 degrees of freedom
Multiple R-squared:  0.5162,    Adjusted R-squared:  0.03233
F-statistic: 1.067 on 8 and 8 DF,  p-value: 0.4647


If you are interested in interaction, try

mod2 <- lm(Y ~ D*F, data=dat)


and you will se what said above. In addition, there will be no degrees of freedom for error, so you will need to replicate the design in this case.