# Kernel Regression

In Kernel Regression with a linear kernel, we have $$\beta = X\alpha = X^T(XX^T+\lambda I)^{-1}Y$$ and the normal Ridge Regression solution is $$\beta = (X^TX+\lambda I)^{-1}X^TY.$$ How can you prove that $X^T(XX^T+\lambda I)^{-1} = (X^TX+\lambda I)^{-1}X^T$?

Starting with

$$(X^TX)X^T + \lambda IX^T = (X^TX)X^T + \lambda IX^T$$

Use the associativity of matrix multiplication, and the fact that multiplication with a diagonal matrix is commutative, to obtain

$$(X^TX)X^T + \lambda IX^T = X^T(XX^T) + X^T\lambda I$$

Using distributivity of addition / multiplication,

$$(X^TX + \lambda I)X^T = X^T(XX^T + \lambda I)$$

for $\lambda > 0$, the matrices in the parentheses are positive definite and have an inverse. Simply multiply both sides on the left by $(X^TX + \lambda I)^{-1}$ and on the right by $(XX^T + \lambda I)^{-1}$.