In Kernel Regression with a linear kernel, we have $$\beta = X\alpha = X^T(XX^T+\lambda I)^{-1}Y$$ and the normal Ridge Regression solution is $$\beta = (X^TX+\lambda I)^{-1}X^TY.$$ How can you prove that $X^T(XX^T+\lambda I)^{-1} = (X^TX+\lambda I)^{-1}X^T$?
0
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Starting with
$$ (X^TX)X^T + \lambda IX^T = (X^TX)X^T + \lambda IX^T $$
Use the associativity of matrix multiplication, and the fact that multiplication with a diagonal matrix is commutative, to obtain
$$ (X^TX)X^T + \lambda IX^T = X^T(XX^T) + X^T\lambda I $$
Using distributivity of addition / multiplication,
$$ (X^TX + \lambda I)X^T = X^T(XX^T + \lambda I) $$
for $\lambda > 0$, the matrices in the parentheses are positive definite and have an inverse. Simply multiply both sides on the left by $(X^TX + \lambda I)^{-1}$ and on the right by $(XX^T + \lambda I)^{-1}$.
