1
$\begingroup$

I'm searching for the correct calculation for a confidence interval using weighted least squares regression. Let me introduce you to my problem.

Guess we have thirteen ordinal classes 1 to 13. For each class there is given a mean of observations belonging to this class with a different number of observations. The assumption is, that these means have got a linear trend over the classes.

Example using R:

xi = 1:13     # classes 
ni = c(163, 455, 1008, 2831, 4652, 7347, 12083, 8395, 9045, 2895, 1192, 1786, 915)   # number of observations in classes
yi = c(-3.98, -3.38, -3.45, -2.96, -2.58, -2.44, -1.91, -1.22, -0.25, -0.43, -0.69,  0.47,  0.37) # means of classes
wi = c(0.003089052, 0.008622814, 0.019102848, 0.053650956, 0.088161161, 0.139234749, 0.228987814, 0.159095647, 0.171413952, 0.054863835, 0.022589876, 0.033846912, 0.017340383) # weights

The weights wi are calculated by $\frac{\sigma^2_i}{n_i}$ and written on the diagonal of matrix $W$, where $\sigma^2_i$ is the variance of the calculated mean of class $i$. So the weighted least square estimation calculated by $\hat{\beta} = (X^tWX)^{-1}X^tWY$ is:

summary(lm(yi ~ xi, weights=wi))

Call:
lm(formula = yi ~ xi, weights = wi)

Weighted Residuals:
     Min       1Q   Median       3Q      Max 
-0.11955 -0.08490  0.01122  0.02580  0.23023 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) -4.89535    0.34246  -14.29 1.89e-08 ***
xi           0.45456    0.04436   10.25 5.79e-07 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.09519 on 11 degrees of freedom
Multiple R-squared:  0.9052,    Adjusted R-squared:  0.8965 
F-statistic:   105 on 1 and 11 DF,  p-value: 5.792e-07

Now, let us assume that the number of obersvations in each class is multiplied with 10. Assuming further we get the same variance $\sigma^2_i$ for each mean of class $i$ as before winew = 0.1 * wi. Then we calculate the new estimation:

summary(lm(yi ~ xi, weights=winew))

Call:
lm(formula = yi ~ xi, weights = winew) 

Weighted Residuals:
      Min        1Q    Median        3Q       Max 
-0.037805 -0.026847  0.003548  0.008159  0.072806 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) -4.89535    0.34246  -14.29 1.89e-08 ***
xi           0.45456    0.04436   10.25 5.79e-07 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.0301 on 11 degrees of freedom
Multiple R-squared:  0.9052,    Adjusted R-squared:  0.8965 
F-statistic:   105 on 1 and 11 DF,  p-value: 5.792e-07

So, as I expected the Residual standard error decreases because of the increasing number of observations.

Can anybody explain me in detail why the standard errors for the estimated coefficients are identical? I thought with an increasing number of observations the standard errors are decreasing.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.