# Eigenvalues in Ridge regression [duplicate]

The ridge regression estimate is given by $$\beta^{*}=(X'X+kI)^{-1}X'y, k≥0,$$ where $$X$$ is the feature matrix. The original paper, Hoerl and Kennard's Ridge Regression: Biased Estimation for Nonorthogonal Problems, states that the eigenvalues, $$\lambda_i$$, of $$X'X$$ are related to eigenvalues, $$\xi_i$$, of $$W = (X'X+kI)^{-1}$$ as $$\xi_i=1/(k+\lambda_i)$$. This expression follows from solving the characteristic equation $$|W - \xi_iI|=0$$. I can only imagine using cofactor representation of the determinant. However, the inverse in $$W$$ complicates matters.

How exactly does one solve this characteristic equation?

If $$\lambda$$ is a nonzero eigenvalue of an invertible matrix $$A$$, then $$\lambda^{-1}$$ is an eigenvalue of $$A^{-1}$$. To show this, note that if $$Av=\lambda v$$ then $$v = A^{-1}(\lambda v) = \lambda A^{-1} v$$ and thus $$\lambda^{-1} v = A^{-1} v$$.
• Just want to point out that we don't need to assume that $\lambda$ is nonzero if we're already assuming that $A$ is invertible, as all eigenvalues of invertible matrices are nonzero. – Eric Perkerson Oct 4 at 19:53