9
$\begingroup$

While reading The Elements of Statistical Learning, I've encountered the term "point-wise variance" several times. While I have a vague idea of what it likely means, I'd be grateful to know

  • How is it defined?
  • How is it derived?
$\endgroup$
  • 1
    $\begingroup$ This typically means the variance of the estimator of a function evaluated at a point. This is, $\mbox{Var}[\hat{f}(x_0)]$. See for example pp. 146. $\endgroup$ – user10525 Oct 29 '12 at 12:34
  • $\begingroup$ Thanks for pointing me towards the definition. I still don't understand - how can a single point have variance? Variance describes deviation from the expectation, so multiple points are needed for such a deviation to be possible, yet evaluating $\hat{f}(x_0)$ gives only one point (?). Is this the variance obtained from estimating the function at $x_0$ over multiple samples from the same population? $\endgroup$ – miura Oct 29 '12 at 13:16
  • 1
    $\begingroup$ Note that the variance is not calculated for $x_0$ but for $\hat{f}(x_0)$. Morever, the estimator $\hat{f}$ is a random variable. An example of this is a kernel density estimator $\hat{f}_h(x_0)=\frac{1}{nh}\sum_{j=1}^n K\left(\frac{x_0-X_j}{h}\right)$ based on a sample $X_1,...,X_n$. Here the variance is calculated with respect to the sample $X_1,...,X_n$ and it can be calculated for each value $x_0$ in the support of the kernel. This is, $\mbox{Var}(\hat{f}(x_0))$ is a function of $x_0$. $\endgroup$ – user10525 Oct 29 '12 at 14:10
  • $\begingroup$ So one could say point-wise variance is equivalent to the standard error of the statistic $\hat{f}(x_0)$, $X_1,...,X_n$ denotes repeated samples, and $Var(\hat{f}(x_0))$ stems from sampling variability? $\endgroup$ – miura Oct 29 '12 at 15:02
  • 1
    $\begingroup$ I agree with your interpretation $\mbox{modulo}$ a square root. $\endgroup$ – user10525 Oct 29 '12 at 15:06
-1
$\begingroup$

On page 267 of ISLR:

What is the variance of the fit, i.e. $\mathrm{Var}(\hat f(x_0))$? Least squares returns variance estimates for each of the fitted coefficients $\hat \beta_j$, as well as the covariances between pairs of coefficient estimates. We can use these to compute the estimated variance of $\hat f(x_0)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.