Effective degrees of freedom for regularized regression If I have a quadratic programming problem
$$\min_b \frac{1}{2} b^tX^tXb - b^t(X^tY)$$
which I regularize by adding a multiple of the identity$\lambda I$ to $X^tX$, then the effective degrees of freedom are
$${\rm d.f.} = \sum_i \frac{s_i}{s_i+\lambda}$$
where the $s_i$ are the singular values of $X$ (equivalently, the eigenvalues of $X^tX$).
But what if I add a multiple of an arbitrary positive definite matrix, $\lambda A$, to $X^tX$? What are the effective degrees of freedom?
 A: Assume $\mathbb A$ is invertible.  Let $\mathbb A = \Sigma^\prime \mathbb U^\prime \mathbb U \Sigma$ where $\mathbb U$ is an orthogonal matrix, $\mathbb {U^\prime U} = \mathbb I$, and write
$$\mathbf \beta = \mathbb U \Sigma \mathbf b$$
so that
$$\mathbf \beta ^\prime \mathbf \beta = \mathbf b^\prime \mathbb A \mathbf b.$$
Then, since $\mathbb U \Sigma$ is invertible,
$$\eqalign{
\frac{1}{2}\mathbf b^\prime \mathbb{X^\prime X} \mathbf b - \mathbf b^\prime \mathbb{X^\prime} \mathbf y &=\frac{1}{2}\left(\left(\mathbb U\Sigma\right)^{-1} \beta\right)^\prime \mathbb{X^\prime X} \left(\left(\mathbb U\Sigma\right)^{-1} \beta\right) - \left(\left(\mathbb U\Sigma\right)^{-1} \beta\right)^\prime \mathbb{X^\prime} \mathbf y \\
&=\frac{1}{2}\mathbf \beta^\prime \mathbb{Z^\prime Z} \mathbf\beta - \mathbf \beta^\prime \mathbb{Z^\prime} \mathbf y
} $$
where
$$\mathbb Z = \mathbb X \Sigma^{-1} \mathbb U^{-1}.$$
The "generalized regularized" objective function can therefore be written
$$\frac{1}{2}\mathbf b^\prime \left(\mathbb{X^\prime X} + \lambda \mathbb A\right) \mathbf b - \mathbf b^\prime \mathbb{X^\prime} \mathbf y =
\frac{1}{2}\mathbf \beta^\prime \left(\mathbb{Z^\prime Z} + \lambda \mathbb I\right) \mathbf\beta - \mathbf \beta^\prime \mathbb{Z^\prime} \mathbf y, $$
which is back in the usual regularized form.  Regardless of what $\mathbb U$ may be, its orthogonality guarantees the eigenvalues of $\mathbb {Z^\prime Z} = \mathbb{U^{-1\prime} \Sigma^{-1\prime} X^\prime X \Sigma ^{-1} U^{-1} }$ will be those of $ \mathbb{\Sigma^{-1\prime} X^\prime X \Sigma ^{-1} }$, whence--writing this common set of eigenvalues as $(t_i)$, the sum
$$\sum_i \frac{t_i}{t_i+\lambda}$$
is well-defined and depends only on $\mathbb A$ and $\mathbb {X^\prime X}$.
