Eigenvalues in Ridge regression [duplicate]

Question

The ridge regression estimate is given by $$\beta^{*}=(X'X+kI)^{-1}X'y, k≥0,$$ where $X$ is the feature matrix. The original paper, Hoerl and Kennard's Ridge Regression: Biased Estimation for Nonorthogonal Problems, states that the eigenvalues, $\lambda_i$, of $X'X$ are related to eigenvalues, $\xi_i$, of $W = (X'X+kI)^{-1}$ as $\xi_i=1/(k+\lambda_i)$. This expression follows from solving the characteristic equation $|W - \xi_iI|=0$. I can only imagine using cofactor representation of the determinant. However, the inverse in $W$ complicates matters.

How exactly does one solve this characteristic equation?

Answer 1

If $\lambda$ is a nonzero eigenvalue of an invertible matrix $A$, then $\lambda^{-1}$ is an eigenvalue of $A^{-1}$. To show this, note that if $Av=\lambda v$ then $v = A^{-1}(\lambda v) = \lambda A^{-1} v$ and thus $\lambda^{-1} v = A^{-1} v$.