Question regarding Optimal Designs of experiments

Question

I'm a bit unclear on the concept of optimal design of a data matrix $X$. I propose a small example to work through:

Suppose $\epsilon_i \sim N(0, \sigma^2)$ are i.i.d., and I have some experiment where we have

$$y_i = x_{1i}\beta_1 + x_{2i}\beta_2 + \epsilon_i,$$ and levels $x_{ij} \in \{-1, 0, 1\}$.

Now, if I represent the above in matrix notation $(Y = X\beta)$, clearly the ols estimator is

$$\hat{\beta} = (X'X)^{-1}X'Y \quad\quad\text{and}\quad\quad \text{Var}(\hat{\beta}) = \sigma^2(X'X)^{-1}.$$

Question: In my toy example then we could use "D-optimality" where we would maximize, $\det\left(\frac{1}{\sigma^2}(X'X)\right)$, with respect to $x_{ij}$. So,

$$ \begin{align} \max_{x_{ij} \in \{-1, 0, 1\}} \det\left(\frac{1}{\sigma^2}(X'X)\right) &= \max_{x_{ij} \in \{-1, 0, 1\}}\frac{1}{\sigma^{4}}\det\left(X'X\right) \\ &= \frac{1}{\sigma^{4}}\max_{x_{ij} \in \{-1, 0, 1\}}\det\left(X'X\right) \\ &= \frac{1}{\sigma^{4}}\max_{x_{ij}\in \{-1, 0, 1\}} (x_{11}x_{22} - x_{12}x_{21}). \end{align} $$

So, the D-optimal $X$ is any such matrix where the determinant of the above is 2, which gives

$$X = \begin{pmatrix} 1 & -1 \\ 1 & 1\end{pmatrix} \quad\quad\text{or}\quad\quad X = \begin{pmatrix} 1 & 1 \\ -1 & 1\end{pmatrix}.$$

Is this correct? The intuitive idea of this at that we gain the most "info" with a design this way or?

Answer 1

Your answer is essentially correct, although is perhaps too small to get any real intuition here.

The two solutions you have correspond to

• keeping $x_2$ fixed and varying $x_1$ as much as possible to get the best possible estimate of $\beta_1$
• keeping $x_1$ fixed and varying $x_2$ as much as possible to get the best possible estimate of $\beta_2$

There are some other solutions which would give a determinant of 2 (-1 multiplied by either of your solutions for example).

Thus we change one factor at a time, and it allows us to get some estimate of one of the unknown parameters, but no estimate of the other.

As we only have two design points (two rows in your matrix), if you were to vary both points at the same time e.g. with a design $\begin{pmatrix}1&1\\-1&-1\end{pmatrix}$, you could not tell if any variation in the response was due to varying $x_1$ or varying $x_2$, and could not estimate $\beta_1$ or $\beta_2$ independently of the other. Thus we would not get any information about the unknown parameters from the experiment, according to the D-optimality criterion.

I would suggest repeating your example, but for $n>2$.