Representation (Notation) of the Design matrix for Linear Regressions

Question

In the section on linear regressions $\mathbf{Y} = \mathbb{X}\beta + \epsilon$, my textbook represents the design matrix as

$\mathbb{X} = \begin{bmatrix} \mathbf{x}_{1}^T \\ \vdots \\ \mathbf{x}_{n}^T \\ \end{bmatrix} = \begin{bmatrix} x_{11} & \dots & x_{1p} \\ \vdots \\ x_{n1} & \dots & x_{np} \end{bmatrix} \in \mathbb{R}^{n \times p}$

I realise that the $T$ in the vector means transpose, but since the transpose operator is on each individual element $\mathbf{x}$ rather than the entire vector itself, what is its function here? Comparing the vector and the matrix, it seems like the transpose operator is doing nothing here; the matrix looks as it would were the $T$ not present. Or am I misunderstanding something?

I would greatly appreciate it if people could please take the time to clarify this.

Answer 1

Each $\mathbf{x}_i$ is a $p \times 1$ matrix: $$ \mathbf{x}_1 = \begin{bmatrix}{x_{11} \\ \ \ \vdots \\ x_{1p}}\end{bmatrix}, \ldots,\ \mathbf{x}_n = \begin{bmatrix}{x_{n1} \\ \ \ \vdots \\ x_{np}}\end{bmatrix}. $$ So each $\mathbf{x}_i^T$ is a $1 \times p$ matrix: $$ \mathbf{x}_1^T = \begin{bmatrix}{x_{11} \ \ldots \ x_{1p}}\end{bmatrix}, \ldots,\ \mathbf{x}_n^T = \begin{bmatrix}{x_{n1} \ \ldots \ x_{np}}\end{bmatrix}. $$ The notation $$ \begin{bmatrix} \mathbf{x}_{1}^T \\ \vdots \\ \mathbf{x}_{n}^T \\ \end{bmatrix} $$ means that the $1 \times p$ matrices are stacked on top of each other, giving the $n \times p$ matrix $$ \begin{bmatrix} x_{11} & \dots & x_{1p} \\ \vdots & & \vdots \\ x_{n1} & \dots & x_{np} \end{bmatrix}. $$

Answer 2

The issue here is only about notation. Usually, in textbooks the vector symbol $\boldsymbol{x}$ represents the column vector of dimension $p \times 1$. By using the transpose, you are effectively considering the design matrix of dimension $n \times p$.

Answer 3

The initial notational convention here is that by $\mathbb x$ we mean a column vector.

Further, in regression analysis (and in most but not all places/fields), and when using matrix algebra notation we use $\mathbb x_i$ to represent the observation vector (i.e. one value from each regressors in the same cross section or time period).

Further, the desing matrix is written as having columns representing each one regressor: the row dimension is the number of observations, and the column dimension is the number of regressors.

Therefore, $\mathbb x_i^T$ represents the full first observation as a row.