Experimental Design with Factorial Block Structure

Question

I'm designing a language-learning study with maybe 30 participants. My collaborators want each person to experience both of the methods we're using, so we are planning a crossover design. In addition to the "method" factor (and a random effect for each person's aptitude), we have two nuisance factors to block by. One is language: we'll use two different languages to reduce bleed-through (effects carrying from one round of treatment into the next in our crossover design). The other is order: half the people do method A followed by method B; half do the reverse.

We could have four types of people:

1. Swedish Method 1 followed by Norwegian Method 2
2. Swedish Method 2 followed by Norwegian Method 1
3. Norwegian Method 1 followed by Swedish Method 2
4. Norwegian Method 2 followed by Swedish Method 1

Or we could have just two types:

1. Norwegian Method 1 followed by Swedish Method 2
2. Norwegian Method 2 followed by Swedish Method 1

I have three related questions (sorry) (thanks).

1. Can you help me conceptualize potential interactions here?
2. Are either of these designs obviously bad for some reason?
3. If not, then which one is more efficient?

In your answer, keep in mind that our goal is inference on which methods are "better," as measured by some yet-to-be-carefully-selected quantitative variables describing learning and learner preferences. I have looked through Gary Oehlert's experimental design textbook (pdf), especially ch. 13, and I read a few suggested questions here on CV.

Answer 1

For a simplified analysis, suppose that the data $Y$ come from this model. To define notation,

• $X_S = 1$ for Swedish, 0 for Norwegian
• $X_M = 1$ for Method 1, 0 for Method 2
• $X_1 = 1$ if measurement was taken first, 0 if second

and $\beta_0$ is the average score for Method 2, Norwegian, second. Suppose average outcomes for the other conditions conform to

$$E[Y(X=x)] = \beta_0 + \beta_M x_M + \beta_S x_S + \beta_1 x_1.$$

If we use four types of people as in the question (one of each) then $X = [\ 1\ |\ x_M\ |\ x_S\ |\ x_1\ ]$ has rank 4, with 8 rows (2 per person) and 4 columns.

• Swedish Method 1 followed by Norwegian Method 2: 1111 and 1000
• Swedish Method 2 followed by Norwegian Method 1: 1011 and 1100
• Norwegian Method 1 followed by Swedish Method 2: 1101 and 1010
• Norwegian Method 2 followed by Swedish Method 1: 1001 and 1110

We have 8 measurements and 4 coefficients to estimate (not counting variance parameters).

If we use only two types of people, language is aliased with time of measurement, so $X$ has rank 3. (The last two columns sum to the first column.)

• Norwegian Method 1 followed by Swedish Method 2: 1101 and 1010
• Norwegian Method 2 followed by Swedish Method 1: 1001 and 1110

We can recover the quantity of interest $\beta_M$ by estimating the model

$$E[Y(X=x)] = \beta_0 + \beta_M x_M + \beta_{N \& 1} x_{N \& 1}$$.

This has one more degree of freedom left over for the error, so all else being equal, it's a tiny bit more efficient.

All else may not be equal. Suppose the true data-generating mechanism takes the form $$E[Y(X=x)] = \beta_0 + \beta_M x_M + \beta_S x_S + \beta_1 x_1 + \gamma x_M x_S x_1,$$ so that people do slightly better when they learn Swedish first by method one. In an analysis neglecting interactions, this should hopefully show up as a slight increase in $\beta_M$; the improvement is proportional to $\gamma Pr(x_S x_1 = 1\ | \ x_M)$. The four-types-of-people design has this property, but the two-types doesn't: if Swedish is always second, we never even see this particular interaction.