# Matrix Orthogonal to Vector: why take transpose?

In econometrics, we often have n observations (in a column vector $$y$$) which we want to explain with k$$<$$n regressors (the observations are in an nxk matrix $$X$$). In this case we use least squares estimation and we can uniquely write the observation as a sum of a fitted value $$\hat{y}\in C(x)$$ and a residual $$\hat{\epsilon} \notin C(X)$$, where $$C(X)$$ is the column span of X: $$y = \hat{y} +\hat{\epsilon}$$

In this case we have $$X^T\hat{\epsilon} = 0$$ which expresses "othogonality". My question is, Where does the transpose on $$X$$ come from? The above form is very reminiscent of that of the orthogonality of two column vectors defined by the inner product: $$ = x^Ty = 0$$, but I have never seen "the inner product" of matrices defined and when we multiply matrices we never need to take the transpose.

The equation $$X^\top \epsilon = 0$$ is simply matrix shorthand for the system of equations \begin{align} x_1^\top \epsilon & = 0 \\ & \vdots \\ x_n^\top \epsilon & = 0 \\ \end{align} where $$x_i$$ is a vector in $$\mathbb{R}^n$$ representing the $$i$$th column of the design matrix $$X$$. This system encodes the constraint that each feature vector be orthogonal to the residual vector.