I am interested in the effect of two different factors, $x_1$ and $x_2$, on a response $y$. However, although I'm interested in both, I'm more interested in the effect of $x_1$ on $y$ than I am in the effect of $x_2$ on $y$. (I am willing to assume that $x_1$ and $x_2$ have independent effects, that the effects are linear, and that there is no interaction between $x_1$ and $x_2$. I also expect both $x_1$ and $x_2$ to have an effect of $y$ based on other evidence.)

Say I can afford to make four observations. I'm considering two scenarios, $A$ and $B$. I'm trying to understand under what circumstances each scenario would be preferred.

In scenario $A$, I run a two-factor, two-level experiment without replication. I assume that $x_1$ and $x_2$ do not interact, and estimate the coefficients $\beta_1$ and $\beta_2$ for the model:

$y(x_1, x_2) = const + \beta_1x_1 + \beta_2x_2 + \epsilon$

where $\epsilon$ is an error term.

In scenario $B$, I run a one-factor, two-level experiment with two replicates (so still four observations total) and just estimate $\beta_1$:

$y(x_1) = const + \beta_1x_1 + \epsilon$

Obviously, the pro for scenario $A$ is that I can estimate both $\beta_1$ and $\beta_2$. But I can't get that $\beta_2$ estimate for free, can I? My question is about what it is scenario $A$ gives up with respect to scenario $B$ in order to be able to estimate two coefficients instead of just one. (In this particular case, I am willing to trade a substantially better estimate of $\beta_1$ for an estimate of $\beta_2$.)

My first thought was that the estimate of $\beta_1$ in scenario $B$ must be more precise, since scenario $B$ has more degrees of freedom left over for the errors. However, if I'm assuming $x_1$ and $x_2$ really are independent, and that $\epsilon$ has the same distribution at all factor levels, then I think I can see why the scenario $A$ estimate of $\beta_1$ could be as good as that from scenario $B$, even though I don't know enough to prove it. After all, in both scenarios, there are two observations for "low" $x_1$ and two for "high" $x_1$.

Also, if I make a little spreadsheet simulation and try both scenarios, I get a really similar histogram of estimate results for $\beta_1$. The picture below is typical of what I get if I simulate using $0$ and $1$ for the high and low values of the the $x$s, $1$ for the $\beta$s, and let the error term be $\epsilon \sim \mathcal N(0, 1)$:

histogram comparison


(1) Is it true that both scenarios will estimate $\beta_1$ with the same precision?

(2) If so, what is scenario $B$ getting out of having two replicates for each level of $x_1$? Perhaps that fewer assumptions are needed, for example about the uniformity of the errors at all factor levels? (Now that could be tested and not just assumed.)

  • 1
    $\begingroup$ I think I've figured out an answer, and also a better statement of the question, but it will be a while before I can post it... $\endgroup$ – jtolle Feb 4 '14 at 2:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.