# Joint asymptotic distribution and measures of fit

I have been looking for something similar but couldn't find any, unfortunately. Some help would be much appreciated.

Consider a simple bivariate linear mean regression $$y=\beta x + e$$

where $$E[e|x]=0,$$ and $$E[x]=0.$$ Suppose we have a random sample $$\{(x_i,y_i)\}_{i=1}^n$$. Let $$\hat{\beta}$$ be an OLS estimator of $$\beta$$. The regression residuals are thus $$\hat{e_i}=y_i-\hat{\beta}x_i$$ for all $$i=1,...,n.$$

What would be the joint asymptotic distribution of the following two measures: $$\hat{\sigma}_y^2=\frac{1}{n}\sum_{i=1}^n y_i^2$$ and $$\hat{\sigma}^2=\frac{1}{n}\sum_{i=1}^n \hat{e}_i^2$$

Based on the previous, what would be the asymptotic distribution of the regression $$R^2$$ and what population value does $$R^2$$ estimate? $$R^2=1-\frac{\hat{\sigma_y^2}}{\hat{\sigma^2}}$$

Update with my idea for a solution

Joint probability is as follows:

For the first measure $$\hat{\sigma}_y^2=\frac{1}{n}\sum_{i=1}^n y_i^2\xrightarrow{p}E[y_i^2]=var(y)$$

$$\sqrt{n}(\hat{\sigma_y^2-var(y)})\xrightarrow[CLT]{d}\mathcal{N}(0,var(y^2)$$

Now for the second measure we can write the following, using LLN: $$\hat{\sigma}^2=\frac{1}{n}\sum_{i=1}^n \hat{e}_i^2= \frac{1}{n}\sum_{i=1}^n(y_i-\beta x_i+\beta x_i-\hat{\beta}x_i)^2$$ $$=\frac{1}{n}\sum_{i=1}^n e_i^2 +(\beta-\hat{\beta})\frac{2}{n}\sum_{i=1}^n e_ix_i+(\beta-\hat{\beta})\frac{1}{n}\sum_{i=1}^n x_i^2$$

From which these terms are calculated as using LLN: $$\frac{1}{n}\sum_{i=1}^n e_i^2\xrightarrow{p}E[e_i^2]$$ $$\frac{1}{n}\sum_{i=1}^n e_ix_i\xrightarrow{p}E[e_ix_i]=0\ from\ LIE\ and\ E[e_i]=0$$ $$\hat{\beta}\xrightarrow{p}\beta$$

To prove consistency we use CMT and have the following: $$\hat{\sigma^2}\xrightarrow{p}\sigma^2$$ because $$E[e_i^2]+0*2*0+0*E[x_i^2]$$

$$\sqrt{n}(\hat{\sigma^2-\sigma^2})=\frac{1}{n}\sum_{i=1}^n(\hat{e_i^2-\sigma^2})=\frac{1}{n}\sum_{i=1}^n(e_i^2-\sigma^2)+\frac{2}{\sqrt{n}}\sum_{i=1}^n(\beta-\hat{\beta})\sum_{i=1}^n e_ix_i +(\beta-\hat{\beta})\frac{1}{\sqrt{n}}\sum_{i=1}^nx_i^2$$

However we know that $$E[e_i-\sigma^2]=0$$ and $$var[e_i-\sigma^2]=var(e^2)$$. Based on these statements we can write the following using CLT:

$$\sqrt{n}\frac{1}{n}\sum_{i=1}^n(e_i^2-\sigma^2)\xrightarrow{d}\mathcal{N}(0,var(e^2))$$

And based on Slutsky's theorem we conclude that $$\sqrt{n}(\hat{\sigma^2}-\sigma^2)\xrightarrow{d}\mathcal{N}(0,var(e^2))$$

Combining both of them: $$\sqrt{n}\begin{pmatrix} \hat{\sigma^2}-\sigma^2 \\ \hat{sigma_y^2}-var(y) \\ \end{pmatrix}=\sqrt{n}\begin{pmatrix} \frac{1}{n} \sum_{i=1}^n(e_i^2-\sigma^2) \\ \frac{1}{n} \sum_{i=1}^n(y_i^2-var(y)) \\ \end{pmatrix}$$ $$\xrightarrow{d}\mathcal{N}\begin{pmatrix} \begin{pmatrix} 0 \\ 0\\ \end{pmatrix} \begin{pmatrix} var(e^2) & cov(e^2(y^2-var(y)) \\ cov(e^2(y^2-var(y)) & var(y^2) \end{pmatrix} \end{pmatrix}$$

For the second part: $$R^2=1-\frac{\hat{\sigma_y^2}}{\hat{\sigma^2}}$$ Now we can start using the delta method, where we express $$g(u_1,u_2)'=1-\frac{u_1}{u_2}$$. The formula is $$\frac{\partial g(u_1,u_2)'}{\partial(u_1,u_2)}=\begin{pmatrix} \frac{-1}{u_2} & \frac{u_1}{u_2^2}) \\ \end{pmatrix}$$ Expressing G in its true values we get: $$G=\begin{pmatrix} \frac{-1}{\sigma_y^2} & \frac{\sigma^2}{\sigma_y^4}) \\ \end{pmatrix}$$ where $$G'=\begin{pmatrix} \frac{-1}{\sigma_y^2} \\ \frac{\sigma^2}{\sigma_y^4}) \\ \end{pmatrix}$$

$$V_{R^2}=G\Sigma G'$$, so we have $$G\Sigma G'=\begin{pmatrix} \frac{-1}{\sigma_y^2} & \frac{\sigma^2}{\sigma_y^4}) \\ \end{pmatrix}\begin{pmatrix} var(e^2) & cov(e^2(y^2-var(y)) \\ cov(e^2(y^2-var(y)) & var(y^2) \end{pmatrix}\begin{pmatrix} \frac{-1}{\sigma_y^2} \\ \frac{\sigma^2}{\sigma_y^4}) \\ \end{pmatrix}$$ $$\sqrt{n}(R^2-R_{POP}^2)\xrightarrow{d}\mathcal{N}(0,V_{R^2})$$ While by doing the algebra we get the following: $$V_{R^2}=\frac{var(e^2)}{var(y)^2}+\frac{\sigma^4 var(y-E[y])^2}{var(y)^4}-\frac{2\sigma^2 cov(e^2,(y-E[y])^2)}{var(y)^3}$$

• The answer to the last question is the easy one: it estimates the population $R^2.$ For the distribution of $R^2,$ see en.wikipedia.org/wiki/… for the exact distribution of $r$ (when there is an intercept term; I believe the solution is simpler in the absence of the intercept) and use $R^2 = r^2.$ Finally, your remaining question about the joint distribution requires some assumptions about the joint distribution of $(x,y)$ in order to make sense of what "asymptotic" might mean.
– whuber
Mar 9, 2021 at 22:33