1
$\begingroup$

In Kernel Regression with a linear kernel, we have $$\beta = X\alpha = X^T(XX^T+\lambda I)^{-1}Y$$ and the normal Ridge Regression solution is $$\beta = (X^TX+\lambda I)^{-1}X^TY.$$ How can you prove that $X^T(XX^T+\lambda I)^{-1} = (X^TX+\lambda I)^{-1}X^T$?

$\endgroup$
1
$\begingroup$

Starting with

$$ (X^TX)X^T + \lambda IX^T = (X^TX)X^T + \lambda IX^T $$

Use the associativity of matrix multiplication, and the fact that multiplication with a diagonal matrix is commutative, to obtain

$$ (X^TX)X^T + \lambda IX^T = X^T(XX^T) + X^T\lambda I $$

Using distributivity of addition / multiplication,

$$ (X^TX + \lambda I)X^T = X^T(XX^T + \lambda I) $$

for $\lambda > 0$, the matrices in the parentheses are positive definite and have an inverse. Simply multiply both sides on the left by $(X^TX + \lambda I)^{-1}$ and on the right by $(XX^T + \lambda I)^{-1}$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.