Degrees of freedom for calculating parameter covariance in weighted least squares

Question

Wikipedia says the standard way to estimate the parameter covariance $M^\beta$ for weighted least squares (WLS) is $$ M^\beta = \chi_\nu^2 (X^TWX)^{-1} $$ with $$ \chi_\nu^2=\frac{r^TWr}{\nu} $$ and the degrees of freedom $$ \nu=n-m $$ (n=number of samples, m=number of fitted parameters, r=residuals, W=diagonal weight matrix, X=data).

My question is about the formula for the degrees of freedom, i.e., the normalization of the reduced chi squared statistic.

Assume I have a bunch of samples with weights 1 and only one with weight 2. Since in WLS, weights correspond to the sample measurement precision, this should be the exact same scenario as simply observing the sample with weight 2 twice (independently).* Thus, WLS can be "emulated" by doing OLS with an extra repeated measurement sample. This yields identical parameter values $\beta$ but, using the above formula for the degrees of freedom, a different $\chi_\nu^2$.

I can achieve equivalence of the two parameter covariance estimates by choosing $$ \nu = \text{sum(diag(}W)) - m, $$ i.e., replacing $n$ in the formula by the sum of the weights. (This obviously reduces to the standard case for all-unit weights.) However, I am confused that the standard formula (which is also the one implemented by, e.g., lm in R and fitlm in Matlab) does not seem to "work" for this simple test case. What am I missing or misunterstanding?

There are many related questions already, e.g., on DOFs or on weighted sample variances.

*Precisions of multiple Gaussian observations are additive... two observations with the same mean and precision 1 are equivalent to one observation with precision 2. The only clear reference for this I could find is this paper, Table 2, Eq. (II.1).

Answer 1

Very good question.

But based on what, do you claim "this should be the exact same scenario"?

An observation (assuming the last WLOG) with a weight of 2, mathematically, means that the disturbance for that observation carries half of the variance. Suppose some higher being offers you a deal where you could have the opportunity to get an extra observation but both the old last and the new observation would get the regular variance. Would you take this deal?

The answer is yes, as there is more information under the new deal. You would do no worse just by taking the average of both x and y over the old last and the new extra. But there is also intra-variability of data between them that carries information (and this is the extra information).

Now suppose you take the deal and, by miracle, the new observation happens to coincide with old last in terms of x and y. Then there is no intra-variability extra information in this particular case (or this particular point in the sample space, if you want to be technical). Then the new point estimate would not change.

But the new estimator is indeed more accurate and should have a smaller standard error, in the classical statistical sense (i.e. frequencist sense across all the "parallel universes", or sample points, that could have been).

Edit:

The above attempted answer, trying to explain intuitively why the OLS/GLS formulae would indicate that two observations with regular variance is more informative than a single observation with half of the variance, is flawed. My claim, that you "can do no worse" by taking the average (and throwing away the extra information), is wrong.

The informational value of an observation increases with a smaller disturbance variance, but also decreases with a smaller variance on the independent variable. So smoothing over the two observations would indeed halve the disturbance variance but would also make the independent variable less insightful. So, if you throw away the extra intra-variability information (e.g. differences within the Xs and Ys), you would be strictly worse off than the original situation of a single observation with half of the variance.

My further attempt to repair this, by utilizing the extra intra-variability information, so far just becomes an alternate way to derive those OLS/WOLS formulae and is no more intuitive than the formulae themselves.

I will think about this more later. Sorry.