Degrees of freedom for a weighted average

Question

And yet another for the long list of degrees-of-freedom questions!

Given an i.i.d. sample $x_1,..., x_n$ from an arbitrary real-valued distribution with expectation $\mu$, the sample mean can be written as $$\bar x = \sum_i \frac{1}{n} x_i.$$ One can say that the degrees of freedom for $\bar x$ is $n-1$, as there are $n$ data points and one parameter being estimated.

Now suppose that some twisted statistician decides that the weights $1/n$ are not good enough, and replaces $\bar x$ with $$\bar x_w = \sum_i w_i x_i,$$ where $w = (w_1, ..., w_n)$ is a pre-determined (i.e., not related to the data) vector of nonnegative weights with $\sum_i w_i = 1$.

What is the degrees of freedom for $\bar x_w$? References to versions of this problem in peer-reviewed work would be of particular interest.

Answer 1

This is wrong: (as correctly pointed out by @zkurtz)

I think the answer given by @zbicyclist as a comment above is quite sensible. One way to rationalize it is as follows. If you arrange your sample $x_1,\ldots,x_n$ and you "regress" on a vector whose $i$-th element is $w_i / \sum w_i^2$ (with no intercept), you get as the estimate $\hat\beta = \sum_iw_ix_i$.

The degrees of freedom of such regression would be the trace of the matrix $X(X'X)^{-1}X'$; replacing $X$ by the column vector whose $i$ element is as given above, those degrees of freedom turn out to be $(\sum_iw_i)^2/\sum_iw_i^2$. Of course if the $w_i$ are constrained to add up to 1, this coincides with your guess of $1/\sum_iw_i^2$.

As references to lend support to such "degrees of freedom" I think it is interesting Hastie-Tibshirani(1990) Generalized Additive Models, Chapman & Hall, section 3.5. (They give alternatives to the trace of the "hat" matrix.)

This may be right:

Hastie-Tibshirani(1990) cited above propose alternative definitions of "degrees of freedom used" in a general non-parametric smoother $\hat{\boldsymbol{x}} = S \boldsymbol{x}$ as follows: i) trace$(S)$, ii) trace$(S^TS)$ and iii) trace$(2S-S^TS)$. They draw on the analogy with a linear model, in which $S = X^T(X^TX)^{-1}X^T$, whose trace is $p$, the number of parameters (throughout I consider the full rank case). Since $S = X^T(X^TX)^{-1}X^T$ is symmetric idempotent, $S^TS$ and $(2S-S^TS)$ are equal to $S$, so the three definitions give the same answer in the linear regression case.

In the case of the question asked, we may consider $\boldsymbol{\hat{x}} = W \boldsymbol{x}$ where $\boldsymbol{\hat{x}}$ is the weighted mean multiplied by a column vector $\boldsymbol{1}$ and $W$ is a symmetric matrix of weights, each of whose rows is equal to the set of weights used.

If we adopt the definition i) above, the number of degrees of freedom used would be 1 (assuming $\sum_iw_i = 1$, if we adopt ii) it would be $n\sum_iw_i^2$. (In the case $w_i = 1/n$ for all $i$ (ordinary average), this produces 1 as it should.)

I find such definition to have some intuitive appeal, but totally aggree with @whuber that the name "degrees of freedom" (used in the smooth or fit) is abuse of language. I do not believe there is a non-controversial definition.

On this topic I have also found interesting Hodges, J. S. and Sargent, D. J. (2001) Counting Degrees of Freedom in Hierarchical and Other Richly-Parameterised Models, Biometrika, vol. 88, p.367-379. There are many other papers dealing with counting "equivalent parameters" (or "degrees of freedom used") in different situations.