Covariance matrix of the estimates when there is interaction

Question

Consider a factorial design with two treatments A and B. Four possible cases are: administer none of them, administer A, administer B and administer both.

By introducing indicator variables $I_A$ and $I_B$ specifying whether treatment A or B, respectively has been administered, we define following linear model for the response variable:

$Y_i = \beta_0+\beta_1 I_{Ai}+\beta_2 I_{Bi}+\beta_3 I_{Ai}I_{Bi}+\epsilon_i$

where $i=1,...,N$, $\epsilon_i \sim N(0,\sigma^2)$, and $Cov(\epsilon_i,\epsilon^*_i)$. For simplicity, let's assume that each option was sampled $N/4$ times.

If $\beta_3=0$, meaning that there is no interaction term: define design matrix $X$. The maximum likelihood estimator can be found as $\boldsymbol{\hat{\beta}} = (X^T X)^{-1}X^TY$, and the variance matrix of the estimates is $Cov(\boldsymbol{\hat{\beta}}) = \sigma^2 (X^T X)^{-1}$. The treatment effect of none is $\beta_0$, A is $\beta_0+\beta_1$, B is $\beta_0+\beta_2$ and A&B is $\beta_0+\beta_1+\beta_2$. Using $Cov(\boldsymbol{\hat{\beta}})$, we can calculate the covariance between these linear combinations of the coefficients.

Can we make a similar analysis when $\beta_3 \neq 0$?

Answer 1

In statistics, it is important to separate elements (or variables) into two groups: Random or Non-random. Random variable has some special characters such as mean, variance. But non-random elements do not have this kind of thing. In you question, effect of the treatment is the unknown constants (parameters), so there is no variance/covariance.

If effect of treatment means its estimate, there is no difference between general covariates and the interaction. You can treat the interaction term as another covariate. GLM theory has no special limitation on design matrix. With interaction, your design matrix $X$ is Nx4 matrix. The estimated covariance matrix (4x4) is $var(\hat \beta) = \hat σ^2(X'X)^{-1}$. Then you can test/estimate anything that can be expressed as $L\beta$, where $L$ is kx4 matrix.

This explanation is from classical statistical aspect. For Bayesian, maybe everything is random.