$Y_i = \beta_0 + \beta_1 X_i + u_i, i = 1,2,\ldots,N$. Model is heteroskedastic. I am using an iterative version of weighted least squares, in which I iteratively perform weighted regression, then fit the resulting residuals using some nonparametric approach, and then get back to weighted regression and so on. Assuming that the process converges, yielding estimates $\hat{\beta}_0,\hat{\beta}_1$, how do I generate an estimate of their variances?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
The regression estimates for a weighted regression model is $$\hat{\beta} = (X'WX)^{-1}X'WY$$
Now, $Var(\hat{\beta}) = (X'WX)^{-1}X'W Var(Y)W'X(X'WX)^{-1}$
Assuming $Var(u_i) = \sigma_i^2$ and $Cov(u_i,u_j)=0$ for $i\ne j$ $$Var(Y) = diag(\sigma_i^2)$$
You can use the below formulation to calculate the variance of the estimates,$$Var(\hat{\beta}) = (X'WX)^{-1}X'W diag(\sigma_i^2) W'X(X'WX)^{-1}$$
-
1$\begingroup$ This answer ignores the uncertainty in $W$ based on the need to estimate $\sigma^2_i$. It's fairly easy to show that the uncertainty in $W$ doesn't matter if $W$ is determined by a parameter $\alpha$ estimable at $\sqrt{n}$ rate, but that might not be enough if $W$ is determined 'using some nonparametric approach'. I believe it is sufficient that $W$ is determined by a parameter $\alpha$ estimable at faster than $n^{1/4}$ rate, and even just consistency may be enough. $\endgroup$ Commented Nov 30, 2020 at 8:15