I have been looking for something similar but couldn't find any, unfortunately. Some help would be much appreciated.

Consider a simple bivariate linear mean regression $$y=\beta x + e$$

where $E[e|x]=0,$ and $E[x]=0.$ Suppose we have a random sample $\{(x_i,y_i)\}_{i=1}^n$. Let $\hat{\beta}$ be an OLS estimator of $\beta$. The regression residuals are thus $\hat{e_i}=y_i-\hat{\beta}x_i$ for all $i=1,...,n.$

What would be the joint asymptotic distribution of the following two measures: $$\hat{\sigma}_y^2=\frac{1}{n}\sum_{i=1}^n y_i^2$$ and $$\hat{\sigma}^2=\frac{1}{n}\sum_{i=1}^n \hat{e}_i^2$$

Based on the previous, what would be the asymptotic distribution of the regression $R^2$ and what population value does $R^2$ estimate? $$R^2=1-\frac{\hat{\sigma_y^2}}{\hat{\sigma^2}}$$

Update with my idea for a solution

Joint probability is as follows:

For the first measure $$\hat{\sigma}_y^2=\frac{1}{n}\sum_{i=1}^n y_i^2\xrightarrow{p}E[y_i^2]=var(y)$$


Now for the second measure we can write the following, using LLN: $$\hat{\sigma}^2=\frac{1}{n}\sum_{i=1}^n \hat{e}_i^2= \frac{1}{n}\sum_{i=1}^n(y_i-\beta x_i+\beta x_i-\hat{\beta}x_i)^2$$ $$=\frac{1}{n}\sum_{i=1}^n e_i^2 +(\beta-\hat{\beta})\frac{2}{n}\sum_{i=1}^n e_ix_i+(\beta-\hat{\beta})\frac{1}{n}\sum_{i=1}^n x_i^2$$

From which these terms are calculated as using LLN: $$\frac{1}{n}\sum_{i=1}^n e_i^2\xrightarrow{p}E[e_i^2]$$ $$\frac{1}{n}\sum_{i=1}^n e_ix_i\xrightarrow{p}E[e_ix_i]=0\ from\ LIE\ and\ E[e_i]=0$$ $$\hat{\beta}\xrightarrow{p}\beta$$

To prove consistency we use CMT and have the following: $$\hat{\sigma^2}\xrightarrow{p}\sigma^2$$ because $E[e_i^2]+0*2*0+0*E[x_i^2]$

$$\sqrt{n}(\hat{\sigma^2-\sigma^2})=\frac{1}{n}\sum_{i=1}^n(\hat{e_i^2-\sigma^2})=\frac{1}{n}\sum_{i=1}^n(e_i^2-\sigma^2)+\frac{2}{\sqrt{n}}\sum_{i=1}^n(\beta-\hat{\beta})\sum_{i=1}^n e_ix_i +(\beta-\hat{\beta})\frac{1}{\sqrt{n}}\sum_{i=1}^nx_i^2$$

However we know that $E[e_i-\sigma^2]=0$ and $var[e_i-\sigma^2]=var(e^2)$. Based on these statements we can write the following using CLT:


And based on Slutsky's theorem we conclude that $$\sqrt{n}(\hat{\sigma^2}-\sigma^2)\xrightarrow{d}\mathcal{N}(0,var(e^2))$$

Combining both of them: $$\sqrt{n}\begin{pmatrix} \hat{\sigma^2}-\sigma^2 \\ \hat{sigma_y^2}-var(y) \\ \end{pmatrix}=\sqrt{n}\begin{pmatrix} \frac{1}{n} \sum_{i=1}^n(e_i^2-\sigma^2) \\ \frac{1}{n} \sum_{i=1}^n(y_i^2-var(y)) \\ \end{pmatrix}$$ $$\xrightarrow{d}\mathcal{N}\begin{pmatrix} \begin{pmatrix} 0 \\ 0\\ \end{pmatrix} \begin{pmatrix} var(e^2) & cov(e^2(y^2-var(y)) \\ cov(e^2(y^2-var(y)) & var(y^2) \end{pmatrix} \end{pmatrix}$$

For the second part: $$R^2=1-\frac{\hat{\sigma_y^2}}{\hat{\sigma^2}}$$ Now we can start using the delta method, where we express $g(u_1,u_2)'=1-\frac{u_1}{u_2}$. The formula is $$\frac{\partial g(u_1,u_2)'}{\partial(u_1,u_2)}=\begin{pmatrix} \frac{-1}{u_2} & \frac{u_1}{u_2^2}) \\ \end{pmatrix}$$ Expressing G in its true values we get: $$G=\begin{pmatrix} \frac{-1}{\sigma_y^2} & \frac{\sigma^2}{\sigma_y^4}) \\ \end{pmatrix}$$ where $$G'=\begin{pmatrix} \frac{-1}{\sigma_y^2} \\ \frac{\sigma^2}{\sigma_y^4}) \\ \end{pmatrix}$$

$V_{R^2}=G\Sigma G'$, so we have $$G\Sigma G'=\begin{pmatrix} \frac{-1}{\sigma_y^2} & \frac{\sigma^2}{\sigma_y^4}) \\ \end{pmatrix}\begin{pmatrix} var(e^2) & cov(e^2(y^2-var(y)) \\ cov(e^2(y^2-var(y)) & var(y^2) \end{pmatrix}\begin{pmatrix} \frac{-1}{\sigma_y^2} \\ \frac{\sigma^2}{\sigma_y^4}) \\ \end{pmatrix}$$ $$\sqrt{n}(R^2-R_{POP}^2)\xrightarrow{d}\mathcal{N}(0,V_{R^2})$$ While by doing the algebra we get the following: $$V_{R^2}=\frac{var(e^2)}{var(y)^2}+\frac{\sigma^4 var(y-E[y])^2}{var(y)^4}-\frac{2\sigma^2 cov(e^2,(y-E[y])^2)}{var(y)^3}$$

  • $\begingroup$ The answer to the last question is the easy one: it estimates the population $R^2.$ For the distribution of $R^2,$ see en.wikipedia.org/wiki/… for the exact distribution of $r$ (when there is an intercept term; I believe the solution is simpler in the absence of the intercept) and use $R^2 = r^2.$ Finally, your remaining question about the joint distribution requires some assumptions about the joint distribution of $(x,y)$ in order to make sense of what "asymptotic" might mean. $\endgroup$
    – whuber
    Mar 9, 2021 at 22:33


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.