# Leave-one-out cross validation for a linear model through the origin

An Introduction to Statistical Learning by James et al. defines the leave-one-out cross-validation estimate for least-squares as:

$$CV_{(n)}=\frac{1}{n}\sum_{i=1}^n \biggr(\frac{y_i-\hat y_i}{1-h_i}\biggr)^2$$

with $$\hat y_i$$ being the $$i$$th fitted value form the original least squares fit, and $$h_i$$ being the leverage defined by $$h_i=\frac{1}{n} +\frac{(x_i- \bar x)^2}{\sum_{i'=1}^n (x_{i'}-\bar x)^2 }$$. I've been able to find and understand the proof using matrices for this (https://robjhyndman.com/hyndsight/loocv-linear-models/), but I'm having trouble translating it to the case of regression through the origin in a meaningful way.

For the model $$Y=\beta X+\epsilon$$ I've found the estimate of $$\beta$$ to be $$\hat \beta=\frac{\sum_{k=1}^n x_{k}y_k}{\sum_{k=1}^n x_{k}^2}$$

and the variance of the residual error would be $$\frac{1}{n}\sum_{i=1}^n (y_i-\hat y_i^{-i})^2$$ where $$y_i^{-i}$$ is the model output when $$(x_i,y_i)$$ are excluded from the training data. I just can't figure out what the next step would be to apply this for least-squares regression through the origin.

• Welcome to CV! In your second equation ($\hat{\beta} = \dots$), is seems as though you are using all data to estimate $\hat{\beta}$. If you want to use LOOCV, you should omit the $i$-th observation from the estimation at each step $i$. – Frans Rodenburg Oct 29 '18 at 3:42
• @Frans Rodenburg Thank you! I should have clarified my definition a bit better, that estimate of $\hat \beta$ corresponds to the model where the entire sample is used, with response $\hat \y_{i}$. Am I correct in assuming that the $\hat \beta^{-i}$ parameter for model with the $i$th observation removed would be the same summation, expect split into two parts with one summing from $k=1$ to $i-1$ and the other from $k=i+1$ to $n$? – sk13 Oct 29 '18 at 18:42