I'm searching for the correct calculation for a confidence interval using weighted least squares regression. Let me introduce you to my problem.
Guess we have thirteen ordinal classes 1 to 13. For each class there is given a mean of observations belonging to this class with a different number of observations. The assumption is, that these means have got a linear trend over the classes.
Example using R:
xi = 1:13 # classes
ni = c(163, 455, 1008, 2831, 4652, 7347, 12083, 8395, 9045, 2895, 1192, 1786, 915) # number of observations in classes
yi = c(-3.98, -3.38, -3.45, -2.96, -2.58, -2.44, -1.91, -1.22, -0.25, -0.43, -0.69, 0.47, 0.37) # means of classes
wi = c(0.003089052, 0.008622814, 0.019102848, 0.053650956, 0.088161161, 0.139234749, 0.228987814, 0.159095647, 0.171413952, 0.054863835, 0.022589876, 0.033846912, 0.017340383) # weights
The weights wi
are calculated by $\frac{\sigma^2_i}{n_i}$ and written on the diagonal of matrix $W$, where $\sigma^2_i$ is the variance of the calculated mean of class $i$. So the weighted least square estimation calculated by $\hat{\beta} = (X^tWX)^{-1}X^tWY$ is:
summary(lm(yi ~ xi, weights=wi))
Call:
lm(formula = yi ~ xi, weights = wi)
Weighted Residuals:
Min 1Q Median 3Q Max
-0.11955 -0.08490 0.01122 0.02580 0.23023
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -4.89535 0.34246 -14.29 1.89e-08 ***
xi 0.45456 0.04436 10.25 5.79e-07 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.09519 on 11 degrees of freedom
Multiple R-squared: 0.9052, Adjusted R-squared: 0.8965
F-statistic: 105 on 1 and 11 DF, p-value: 5.792e-07
Now, let us assume that the number of obersvations in each class is multiplied with 10. Assuming further we get the same variance $\sigma^2_i$ for each mean of class $i$ as before winew = 0.1 * wi
. Then we calculate the new estimation:
summary(lm(yi ~ xi, weights=winew))
Call:
lm(formula = yi ~ xi, weights = winew)
Weighted Residuals:
Min 1Q Median 3Q Max
-0.037805 -0.026847 0.003548 0.008159 0.072806
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -4.89535 0.34246 -14.29 1.89e-08 ***
xi 0.45456 0.04436 10.25 5.79e-07 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.0301 on 11 degrees of freedom
Multiple R-squared: 0.9052, Adjusted R-squared: 0.8965
F-statistic: 105 on 1 and 11 DF, p-value: 5.792e-07
So, as I expected the Residual standard error decreases because of the increasing number of observations.
Can anybody explain me in detail why the standard errors for the estimated coefficients are identical? I thought with an increasing number of observations the standard errors are decreasing.