2
$\begingroup$

From the general linear model, we know the following :

$E(\pmb{\hat{\beta}}) = E((\mathbf{X}^T \mathbf{X})^{-1}\mathbf{X}^T\mathbf{Y}) $

and

$Var(\pmb{\hat{\beta}}) = \sigma^2(\mathbf{X}^T \mathbf{X})^{-1}$.

Does that hold true when interaction effects are included, that is $\mathbf{X}$ contains variables and the products of each other?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
6
$\begingroup$

Yes. $\mathbf X$ is agnostic about what goes into it; the columns can represent (1) individual numeric covariates (or nonlinear transformations thereof), (2) dummy variables associated with categorical variables, (3) elements of a computed basis (such as a spline or orthogonal polynomial), or (4) elements of the interactions (== products of columns) of any of the above.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you. Would the convergence varies depending on the distributions? $\endgroup$
    – POC
    Commented Oct 8, 2023 at 23:10
  • $\begingroup$ The general linear model typically assumes a Gaussian conditional distribution. Other conditional distributions would suggest that you're working with a generalized (rather than general) linear model, at which points questions about convergence etc. could get much more complicated. $\endgroup$
    – Ben Bolker
    Commented Oct 9, 2023 at 0:04
  • $\begingroup$ @POC convergence of what, exactly? $\endgroup$
    – Glen_b
    Commented Oct 9, 2023 at 2:32
  • $\begingroup$ The composition of X does affect convergence of the fitting algorithm in generalized linear models, because of collinearity. Note that this is not the same as convergence of a sample to any distribution, and also doesn't quite apply to the formula you posted, because in that case you have to worry that $X^TX$ is invertible (there is no multi-step fitting algorithm, and thus no convergence or divergence). $\endgroup$
    – carlo
    Commented Oct 9, 2023 at 9:15
  • $\begingroup$ Maybe the word i'm looking for is efficient rather than convergence @Glen_b. Would the efficiency be the same for all $\mathbf{B}$ if the distributions in $\mathbf{X}$ are different (like the interaction terms)? Maybe, it should be a question altogether. $\endgroup$
    – POC
    Commented Oct 9, 2023 at 12:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.