Steps of Matrix Multiplication It may seem kind of silly, but can anyone please show me the intermediate steps implied by the second equality in this derivation?
$$e^\prime e = \left(y - Xb\right)^\prime\left(y - Xb\right) = y^\prime y - 2b^\prime X^\prime y + b^\prime X^\prime X b.$$
 A: Quite a few rules  are actually needed for this derivation: almost all the algebraic (ring) axioms for real numbers, the axioms for real matrices as a (right) algebra over the real numbers, and properties of transposition as an anti-isomorphism of that algebra.  The only numerical facts needed are used in the last step.
It may be of interest that no details of matrix multiplication are needed for this derivation, showing that it holds in more general statistical settings wherever the same axioms apply.  For instance, real numbers can be replaced by complex numbers; transposition may be replaced by Hermitean conjugation; vectors may be replaced by (certain spaces of) functions and matrices by linear operators.  All have applications, but there is no need to memorize or re-prove this result in all the different cases.
Here is a sketch of the details, assuming $e$ is a vector (as is evident from the form of the right hand side).  (In a multivariate least squares model $e$ would be a matrix in its own right, but then a slightly different result would hold.)


*

*$(y-Xb)^\prime = y^\prime - (Xb)^\prime.$ Transposition is a linear operation.

*$(Xb)^\prime = b^\prime X^\prime.$ Transposition is an anti-isomorphism.

*$(y^\prime - b^\prime X^\prime)(y-Xb) = y^\prime (y-Xb) - (b^\prime X^\prime) (y-Xb).$  This is because $(B-C)A = B A - C A$ for any matrices $A,B,C$ where all operations are defined. Matrix multiplication is right-distributive.  In the application, $A=y-Xb.$

*Similarly, $y^\prime (y-Xb) = y^\prime y - y^\prime (Xb)$ and $(b^\prime X^\prime)(y-Xb) = (b^\prime X^\prime)y - (b^\prime X^\prime)(Xb)$ because matrix multiplication is left-distributive.

*$y^\prime(Xb)$ is a $1\times 1$ matrix and therefore equals its own transpose, $y^\prime (Xb) = (y^\prime(Xb))^\prime.$

*As in $(2),$ $(y^\prime(Xb))^\prime = (Xb)^\prime (y^\prime)^\prime.$

*$(y^\prime)^\prime = y.$ Transposition is an involution.

*Applying $(2)$ and $(7)$ to $(5)$ and $(6)$ yields $(y^\prime (Xb))^\prime = (b^\prime X^\prime)y.$

*Substituting $(8)$ into the first term in $(4)$ and then substituting that result into $(3)$ produces $$(y-Xb)^\prime(y-Xb) = (y^\prime y - (b^\prime X^\prime) y) - ((b^\prime X^\prime)y - (b^\prime X^\prime)(Xb)).$$

*By definition, a matrix difference $A-B$ means the linear combination $(1)A + (-1)B.$ 
In light of this, the distributive, associative, and commutative laws of scalar multiplication (which parallel the same laws for multiplying real numbers) allow the right hand side of $(9)$ to be written $$y^\prime y - (1+1)(b^\prime X^\prime)y + (-1)(-1)(b^\prime X^\prime)(Xb).$$

*Several applications of the associative law of matrix multiplication show that parentheses in any sequence of matrix products can be dropped, because it doesn't matter how the products are grouped provided the order remains the same.  Thus, for instance, $(b^\prime X^\prime)(Xb)$ may unambiguously be written $b^\prime X^\prime X b.$

*Finally, $1+1=2$ and $(-1)(-1)=1,$ whence $(10)$ can be simplified to $$y^\prime y - 2b^\prime X^\prime y + b^\prime X^\prime X b.$$
A: $MSE=(y-Xb)^T(y-Xb)=(y^T-b^TX^T)(y-Xb)=y^Ty-y^TXb-b^TX^Ty+b^TX^TXb$.
Then, note that each term here is scalar, and so $y^TXb$, i.e. ($(1\times n)(n \times k)(k \times 1)$), where $n$ is number of samples (data points), $k$ is number of regressors, including bias term.
For scalar terms, we can either take the transpose or not, i.e. $\alpha=\alpha^T$. So, $y^TXb=(y^TXb)^T=b^TX^Ty$, which is the third term above. Then, MSE becomes $y^Ty-2b^TX^Ty+b^TX^TXb$.
Note: If you give additional information about where you confronted this equation (e.g. linear regression lecture etc.), it'd better suit this forum; otherwise, this question may also suit to math forum (maybe better), although the procedure is quite common in ML for MSE calculation.
