Skip to main content
Post Closed as "Duplicate" by whuber
fixes to grammar, formatting
Source Link
einar
  • 4.3k
  • 2
  • 25
  • 39

RidgeThe ridge regression estimate is given by $\beta^{*}=(X'X+kI)^{-1}X'y, k≥0$, where $$\beta^{*}=(X'X+kI)^{-1}X'y, k≥0,$$ where $X$ is featuresthe feature matrix. OriginalThe original paper, paperHoerl and Kennard's Ridge Regression: Biased Estimation for Nonorthogonal Problems by Hoerl, Kennard states that the eigenvalues, $\lambda_i$, of $X'X$ are related to eigenvalues, $\xi_i$, of $W = (X'X+kI)^{-1}$ as $\xi_i=1/(k+\lambda_i)$. This expression follows from solving the characteristic equation $|W - \xi_iI|=0$. I can only imagine using cofactor representation of the determinant. However, the inverse in $W$ complicates matters.

How exactly does one is supposed to solve this characteristic equation?

Ridge regression estimate is given by $\beta^{*}=(X'X+kI)^{-1}X'y, k≥0$, where $X$ is features matrix. Original paper by Hoerl, Kennard states that eigenvalues $\lambda_i$ of $X'X$ are related to eigenvalues $\xi_i$ of $W = (X'X+kI)^{-1}$ as $\xi_i=1/(k+\lambda_i)$. This expression follows from solving characteristic equation $|W - \xi_iI|=0$. I can only imagine using cofactor representation of determinant. However, the inverse in $W$ complicates matters.

How exactly one is supposed to solve this characteristic equation?

The ridge regression estimate is given by $$\beta^{*}=(X'X+kI)^{-1}X'y, k≥0,$$ where $X$ is the feature matrix. The original paper, Hoerl and Kennard's Ridge Regression: Biased Estimation for Nonorthogonal Problems, states that the eigenvalues, $\lambda_i$, of $X'X$ are related to eigenvalues, $\xi_i$, of $W = (X'X+kI)^{-1}$ as $\xi_i=1/(k+\lambda_i)$. This expression follows from solving the characteristic equation $|W - \xi_iI|=0$. I can only imagine using cofactor representation of the determinant. However, the inverse in $W$ complicates matters.

How exactly does one solve this characteristic equation?

Source Link

Eigenvalues in Ridge regression

Ridge regression estimate is given by $\beta^{*}=(X'X+kI)^{-1}X'y, k≥0$, where $X$ is features matrix. Original paper by Hoerl, Kennard states that eigenvalues $\lambda_i$ of $X'X$ are related to eigenvalues $\xi_i$ of $W = (X'X+kI)^{-1}$ as $\xi_i=1/(k+\lambda_i)$. This expression follows from solving characteristic equation $|W - \xi_iI|=0$. I can only imagine using cofactor representation of determinant. However, the inverse in $W$ complicates matters.

How exactly one is supposed to solve this characteristic equation?