Augmenting experiment design matrices

Question

Consider a set of users (rows) where we are already testing several treatments (columns):

$$\begin{pmatrix} E & U & E \\ U & E &E \\ \vdots & \vdots &\vdots \\ E & E & U\end{pmatrix}$$

For example, if element (j, k) = E that means that, currently user #j is exposed to treatment #k. (E and U mean exposed and unexposed respectively).

Currently, every treatment has 50% of exposed users and 50% of unexposed users (i.e. the # of E's and U's in every column is ~ the same).

Now, say that we decide to test another treatment in parallel. How can we design a new column with a similar split (50% E and 50% U) so that, when we measure effects for any of the treatments (sum of all Es - sum of all Us for that new column), we are minimizing cofounding with other treatments?

Answer 1

Consider the coding $U=-1$ and $E=1$.

If the existing design has orthogonal columns then the columns may be contained in a Hadamard matrix. You can search over the remaining columns of that matrix to find the D-optimal new column (maximize $|\mathbf{X}^\text{T} \mathbf{X}|$ where $X$ is the model matrix from the first order model) to add to your design. Note that the "optimal" column need not be unique. Since Hadamard matrices are square this should be easy to accomplish computationally.

If no such Hadamard matrix exists then you might just have to search over the $\binom{\text{row count}}{\text{row count}/2}$ possible column arrangements. That may not be computationally feasible.

You can do something like an interior point method and just search over randomly generated columns with entries generated according to $\mathrm{Unif}(-1,1)$, and then use numerical optimization with those columns as starting points to find columns that are D-optimal. If the space of possible columns is "small" then this often finds an optimal column, but if the space is "large" it can fail to find an optimal column but still leave you with a good column. You can generate such a D-optimal augmentation in commercial software like JMP very easily.