Can we use the trace of the hat matrix in case of kernel regression for the effective degrees of freedom? Where we obtain the hat matrix, H as:
$H = \hat{y} * y^+$ where $\hat{y}$ is the predicted response values, $y^+$ is the pseudo inverse of $y$.
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
0
If $H$ is such that $\hat y = H y$, then it would be quite standard to obtain the effective degrees of freedom from $\text{tr}(H)$.
See Hastie and Tibshirani's Generalized Additive Models, and there's a similar treatment in Hastie et al Elements of Statistical Learning (2nd ed) in chapter 5 (In the section on Degrees of Freedom and Smoother Matrices). The discussion in in that section of ESL II is on effective degrees of freedom in smoothing splines but as they say there (where they're specifically discussing such a definition of generalized degrees of freedom given in eq 5.16) "This very useful definition allows us a more intuitive way to parameterize the smoothing spline, and indeed many other smoothers as well, in a consistent fashion"; they then mention that there are many possible justifications for this and they go on to give some of the possible justifications. See also equation 7.6 and the nearby text.
Considerably more broadly, look at Ye (1998) "On Measuring and Correcting the Effects of Data Mining and Model Selection", JASA 93(441):120-131 where the author proposes using $\sum_i \frac{\partial \hat{y}_i}{\partial y_i}$ as the degrees of freedom for a huge variety of models for which you can take a set of data $y$ and compute a $\hat{y}$. For a model which can be written in hat-matrix form $\hat y = H y$, that's the trace of $H$.
