Notation: covariance on scalars? I've recently seen the degrees-of-freedom for K-nearest neighbors regression specified like so:
$\frac{1}{\sigma^2}\sum_{i=1}^NCov(y_i,\hat{y}_i)$ 
But what does $Cov(y_i,\hat{y}_i)$ mean?  Previously I've understood the notation $Cov(X,Y)$ to mean the covariance of the vectors $X$ and $Y$, resulting in a scalar.  Here, the inputs $y_i$ and $\hat{y}_i$ are themselves scalars.  I am confused.
 A: This is the standard definition of the covariance:
$$
\newcommand{\Cov}{\mathrm{Cov}}
\newcommand{\E}{\mathbb{E}}
\Cov(X, Y) = \E\left[ \left( X - \E[X] \right) \left( Y - \E[Y] \right) \right]
,$$
where here $X$ and $Y$ are real-valued random variables.
With an $n$-dimensional random vector $X$, the typical notation is
$$
\Cov(X) = \left[ \Cov(X_i, X_j) \right]_{ij}
,$$
i.e. you get an $n \times n$ matrix where the $(i, j)$th entry is the covariance between $X_i$ and $X_j$.
I'm not familiar with a notation $\Cov(X, Y)$, where $X$ and $Y$ are random vectors and the covariance is a scalar.
A: I guess that $y_i$ and $ \hat{y}_i$ are vectors with respective means $ \bar{y}_i $ and $\hat{\bar{y}}_i$ respectively. 
Maybe they should have specified $ 1/\sigma^2 $ $\sum\limits_{i=1}^n$ ($y_i$ - $ \bar{y}_i $)($ \hat{y}_i$ - $\hat{\bar{y}}_i$). 
I will look into K-nearest neighbors regression and get back as I don't know the underlying theory.
A: It follows from the definition of the effective degrees of freedom given by the trace of the hat matrix, $\nu=\mathrm{tr}(H)$; the diagonal of $H$ equals the ratio of the covariance between $\hat{y}$ and $y$ and the variance of $y$ (link).
