Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

 5 added two tags, ls and regression | link edited Aug 4 '15 at 5:40 Christoph Hanck 19.3k33 gold badges4646 silver badges8282 bronze badges Tweeted twitter.com/#!/StackStats/status/628216262427025408 occurred Aug 3 '15 at 14:50 4 added 75 characters in body edited Aug 1 '15 at 16:31 Clarinetist 3,17811 gold badge1515 silver badges4848 bronze badges From An Introduction to Statistical Learning by James et al., the leave-one-out cross-validation (LOOCV) estimate is defined by $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\text{MSE}_i$$ where $$\text{MSE}_i = (y_i-\hat{y}_i)^2$$. Without proof, equation (5.2) states that for a least-squares or polynomial regression (whether this applies to regression on just one variable is unknown to me), $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\left(\dfrac{y_i - \hat{y}_i}{1-h_i}\right)^2$$ where "$$\hat{y}_i$$ is the $$i$$th fitted value from the original least squares fit (no idea what this means, by the way, does it mean from using all of the points in the data set?) and $$h_i$$ is the leverage" which is defined by $$h_i = \dfrac{1}{n}+\dfrac{(x_i - \bar{x})^2}{\sum\limits_{j=1}^{n}(x_j - \bar{x})^2}\text{.}$$ How does one prove this? My attempt: one could start by noticing that $$\hat{y}_i = \beta_0 + \sum\limits_{i=1}^{k}\beta_k X_k + \text{some polynomial terms of degree }\geq 2$$ but apart from this (and if I recall, that formula for $$h_i$$ is only true for simple linear regression...), I'm not sure how to proceed from here. From An Introduction to Statistical Learning by James et al., the leave-one-out cross-validation (LOOCV) estimate is defined by $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\text{MSE}_i$$ where $$\text{MSE}_i = (y_i-\hat{y}_i)^2$$. Without proof, equation (5.2) states that for a least-squares or polynomial regression, $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\left(\dfrac{y_i - \hat{y}_i}{1-h_i}\right)^2$$ where "$$\hat{y}_i$$ is the $$i$$th fitted value from the original least squares fit (no idea what this means, by the way, does it mean from using all of the points in the data set?) and $$h_i$$ is the leverage" which is defined by $$h_i = \dfrac{1}{n}+\dfrac{(x_i - \bar{x})^2}{\sum\limits_{j=1}^{n}(x_j - \bar{x})^2}\text{.}$$ How does one prove this? My attempt: one could start by noticing that $$\hat{y}_i = \beta_0 + \sum\limits_{i=1}^{k}\beta_k X_k + \text{some polynomial terms of degree }\geq 2$$ but apart from this (and if I recall, that formula for $$h_i$$ is only true for simple linear regression...), I'm not sure how to proceed from here. From An Introduction to Statistical Learning by James et al., the leave-one-out cross-validation (LOOCV) estimate is defined by $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\text{MSE}_i$$ where $$\text{MSE}_i = (y_i-\hat{y}_i)^2$$. Without proof, equation (5.2) states that for a least-squares or polynomial regression (whether this applies to regression on just one variable is unknown to me), $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\left(\dfrac{y_i - \hat{y}_i}{1-h_i}\right)^2$$ where "$$\hat{y}_i$$ is the $$i$$th fitted value from the original least squares fit (no idea what this means, by the way, does it mean from using all of the points in the data set?) and $$h_i$$ is the leverage" which is defined by $$h_i = \dfrac{1}{n}+\dfrac{(x_i - \bar{x})^2}{\sum\limits_{j=1}^{n}(x_j - \bar{x})^2}\text{.}$$ How does one prove this? My attempt: one could start by noticing that $$\hat{y}_i = \beta_0 + \sum\limits_{i=1}^{k}\beta_k X_k + \text{some polynomial terms of degree }\geq 2$$ but apart from this (and if I recall, that formula for $$h_i$$ is only true for simple linear regression...), I'm not sure how to proceed from here. 3 deleted 172 characters in body edited Aug 1 '15 at 16:25 Clarinetist 3,17811 gold badge1515 silver badges4848 bronze badges From An Introduction to Statistical Learning by James et al., the leave-one-out cross-validation (LOOCV) estimate is defined by $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\text{MSE}_i$$ where $$\text{MSE}_i = (y_i-\hat{y}_i^{(i)})^2$$$$\text{MSE}_i = (y_i-\hat{y}_i)^2$$. I am using $$(i)$$ to specify that the $$i$$th fitted model is being used (since you generate $$n$$ different fitted models when doing LOOCV). Without proof, equation (5.2) states that for a least-squares or polynomial regression, $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\left(\dfrac{y_i - \hat{y}_i^{OLS}}{1-h_i}\right)^2$$$$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\left(\dfrac{y_i - \hat{y}_i}{1-h_i}\right)^2$$ where "$$\hat{y}_i^{OLS}$$$$\hat{y}_i$$ is the $$i$$th fitted value from the original least squares fit (no idea what this means, by the way, does it mean from using all of the points in the data set?) and $$h_i$$ is the leverage" which is defined by $$h_i = \dfrac{1}{n}+\dfrac{(x_i - \bar{x})^2}{\sum\limits_{j=1}^{n}(x_j - \bar{x})^2}\text{.}$$ How does one prove this? My attempt: one could start by noticing that $$\hat{y}^{(i)}_i = \beta_0^{(i)} + \sum\limits_{i=1}^{k}\beta_k^{(i)} X_k + \text{some polynomial terms of degree }\geq 2$$$$\hat{y}_i = \beta_0 + \sum\limits_{i=1}^{k}\beta_k X_k + \text{some polynomial terms of degree }\geq 2$$ but apart from this (and if I recall, that formula for $$h_i$$ is only true for simple linear regression...), I'm not sure how to proceed from here. From An Introduction to Statistical Learning by James et al., the leave-one-out cross-validation (LOOCV) estimate is defined by $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\text{MSE}_i$$ where $$\text{MSE}_i = (y_i-\hat{y}_i^{(i)})^2$$. I am using $$(i)$$ to specify that the $$i$$th fitted model is being used (since you generate $$n$$ different fitted models when doing LOOCV). Without proof, equation (5.2) states that for a least-squares or polynomial regression, $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\left(\dfrac{y_i - \hat{y}_i^{OLS}}{1-h_i}\right)^2$$ where "$$\hat{y}_i^{OLS}$$ is the $$i$$th fitted value from the original least squares fit and $$h_i$$ is the leverage" which is defined by $$h_i = \dfrac{1}{n}+\dfrac{(x_i - \bar{x})^2}{\sum\limits_{j=1}^{n}(x_j - \bar{x})^2}\text{.}$$ How does one prove this? My attempt: one could start by noticing that $$\hat{y}^{(i)}_i = \beta_0^{(i)} + \sum\limits_{i=1}^{k}\beta_k^{(i)} X_k + \text{some polynomial terms of degree }\geq 2$$ but apart from this (and if I recall, that formula for $$h_i$$ is only true for simple linear regression...), I'm not sure how to proceed from here. From An Introduction to Statistical Learning by James et al., the leave-one-out cross-validation (LOOCV) estimate is defined by $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\text{MSE}_i$$ where $$\text{MSE}_i = (y_i-\hat{y}_i)^2$$. Without proof, equation (5.2) states that for a least-squares or polynomial regression, $$\text{CV}_{(n)} = \dfrac{1}{n}\sum\limits_{i=1}^{n}\left(\dfrac{y_i - \hat{y}_i}{1-h_i}\right)^2$$ where "$$\hat{y}_i$$ is the $$i$$th fitted value from the original least squares fit (no idea what this means, by the way, does it mean from using all of the points in the data set?) and $$h_i$$ is the leverage" which is defined by $$h_i = \dfrac{1}{n}+\dfrac{(x_i - \bar{x})^2}{\sum\limits_{j=1}^{n}(x_j - \bar{x})^2}\text{.}$$ How does one prove this? My attempt: one could start by noticing that $$\hat{y}_i = \beta_0 + \sum\limits_{i=1}^{k}\beta_k X_k + \text{some polynomial terms of degree }\geq 2$$ but apart from this (and if I recall, that formula for $$h_i$$ is only true for simple linear regression...), I'm not sure how to proceed from here. 2 edited tags | link edited Aug 1 '15 at 7:03 Clarinetist 3,17811 gold badge1515 silver badges4848 bronze badges 1 asked Aug 1 '15 at 5:03 Clarinetist 3,17811 gold badge1515 silver badges4848 bronze badges