Consider the standard simple linear regression model: $$ Y_i = \beta_0 + \beta_1 X_i + \epsilon_i, $$ for $i=1,\dots,n$. In matrix-vector form this is $$ \mathbf{Y} = \mathbf{X_n}\beta + \epsilon, $$ where, in particular, $$ \mathbf{X_n} = \begin{bmatrix} 1 & X_1, \\ 1 & X_2, \\ \vdots & \vdots \\ 1 & X_n \end{bmatrix}. $$
In the case of simple linear regression, the determinant of the matrix $\mathbf{X_n}^T\mathbf{X_n}$ is $$ \text{Det}(\mathbf{X_n}^T\mathbf{X_n}) = n \sum_{i=1}^n(X_i - \overline X_n)^2, $$ where $\overline X_n = \frac{1}{n} \sum_{i=1}^n X_i$.
Now consider multiple regression with $p$ regressors, in which case: $$ \mathbf{X_n} = \begin{bmatrix} 1 & X_{11} & X_{12} & \dots & X_{1p} \\ 1 & X_{21} & X_{22} & \dots & X_{2p} \\ \vdots & \vdots & \vdots & \ddots \vdots \\ 1 & X_{n1} & X_{n2} & \dots & X_{np} \end{bmatrix}. $$
Each rows is an iid observation of the (population) covariates. While each row corresponds to an iid observation of the covariates, the random variables in each row can be dependent on one another.
So for the matrix $\mathbf{X_n}$ in the case of multiple regression:
- Is there a known formula for the determinant of $\text{Det}(\mathbf{X_n}^T\mathbf{X_n})$? Of course, the general formula for the determinant of $(p+1)\times (p+1)$ matrix is one such formula, but I am wondering if there is something 'nicer'? The formula for simple linear regression is really nice because it is just a sum over $(X_i - \overline X_n)^2$, is there something analogous for the case of multiple regression? Or at the very least is there something nicer than the general determinant formula?
- Is $\text{Det}(\mathbf{X_n}^T\mathbf{X_n})$ guaranteed to be positive like it was in the case of simple linear regression?
- Note that in the case of simple linear regression $$\lim_{n \to \infty}\frac{1}{n^{2}}E[\text{Det}(\mathbf{X_n}^T\mathbf{X_n})] = \lim_{n \to \infty}\frac{1}{n} \sum_{i=1}^n(X_i - \overline X_n)^2 = \lim_{n \to \infty}\frac{n-1}{n}\sigma^2 \to \sigma^2.$$ Now, for the multiple regression case, what does $\frac{1}{n^{p+1}}E[\text{Det}(\mathbf{X_n}^T\mathbf{X_n})]$ converge to as $n \to \infty$? Does it convergerge to some population statistic like the simple regression case? Note, we need the scaling to prevent it blowing up, see the comment by jld below.