5
$\begingroup$

Using the method of moments, one can try to approximate the sum of $\chi_{r}^{2}$ variables as $\sum a_{i}Y_{i}$ by equating the $n$-th movements of the sample with the $n$-th movement of the population, and "solve" the parameters this way. However I am stuck in the derivation of Satterthwaite appproximation. The author (Berger&Casella) suggested the following(page 314):

"..to do this we must match second moments, and we need $$\mathrm{E}\left(\sum^{k}_{i=1}a_{i}Y_{i}\right)^{2}=\mathrm{E}\left(\frac{\chi_{v}^{2}}{v}\right)^{2}=\frac{2}{v}+1$$

Applying the method of moments, we can drop the first expectation and solve for $v$, yielding $$\overline{v}=\frac{2}{\left(\sum^{k}_{i=1}a_{i}Y_i\right)^{2}-1}$$"

My naive question is why we can drop the expectation at all? This is not clear from the author's description of the method of movements, in which one merely equate $$m_{j}=\frac{1}{n}\sum^{n}_{i=1}X_{i}^{j}\text{ with } EX^{j}$$ And it seems clear to me that the expectation sign cannot be dropped. Similarly in the last step of the derivation of the approximation formula, the author suggested:

"..substituting this expression for the variance and removing the expectations, we obtain...."(page 315)

Can anyone give a hint? Sorry the question is really "low".

Edit:

A fellow at here suggested that the method of moments assume $E(Z)=Z$ because one equals the two moments. I do not think this follows straightaway from the definition. Even when $j=1$ one has to equate $\frac{1}{n}\sum^{n}_{i=1}X_{i}$ with $EX^{1}$. I do not think this implies $E(Z)=Z$ in general, such that one can use $Z=\sum a_{i}Y_{i}$.

$\endgroup$
  • 2
    $\begingroup$ When the $X$'s have the same distribution, $E(\frac{1}{n}\sum^{n}_{i=1}X_{i}) = E(X_j)$ for all $j$ (from basic properties of expectations). When $n=1$ and $j=1$, you equate $E(X_1)$ with $X_1$. Let $Z=X_1$ and by C&B's definition $E(Z)=Z$, ... and so on. $\endgroup$ – Glen_b Aug 19 '13 at 1:59
  • $\begingroup$ @Glen_b: Thanks for the enlightenment. Now I see everything. $\endgroup$ – Bombyx mori Aug 19 '13 at 2:41
  • $\begingroup$ @Glen_b: I realized I do not know how to end this thread. Thanks for the reminder. $\endgroup$ – Bombyx mori Aug 19 '13 at 3:05
4
$\begingroup$

Background: Understanding the method of moments (MoM) in a basic way

Motivation for the method: The (strong) Law of Large Numbers (LLN) gives us reason to think that (at least for large samples), a sample expectation will be close to the population expectation (note that the LLN applies to higher moments by taking $Z=X^j$). Thus, if we have $iid$ $X_i, i=1,\ldots,n$ we have Casella & Berger's $m_j = \frac{1}{n} \sum_{i=1}^n X_i^j$ is set equal to $\text{E}(m_j) = \text{E}(X_i^j) = \mu_j$.

Why you only need consider first moments: Consider Casella & Berger's $m_j = \frac{1}{n} \sum_{i=1}^n X_i^j$ and note that (as we did in the motivating argument), for any $j$ we can just take $Z_i = X_i^j$ and be left with $m_1$ for a different random variable. That is, all MoM estimators can be thought of as first moment MoMs; we can simply make that substitution to get any other moment we need. So MoM is really just setting $m=\mu$ where $m = \frac{1}{n} \sum_{i=1}^n X_i$ for some set of $iid$ $X_i \sim f_X$.

Why you can think of MoM as 'drop expectations': (i) Take $Z = \frac{1}{n} \sum_{i=1}^n X_i$ and note that $\text{E}(Z)=\text{E}(X)$ by linearity of expectation, so MoM simply takes $Z=\text{E}(Z)$. Similarly, taking $Z^j = \text{E}(Z^j)$ follows immediately from the argument we already used - i.e. we can think of MoM as 'drop expectations', and it will be reasonable because we have some random variable which will be close to its expectation; (ii) more generally, we could reasonably do this ('drop expectations') for any $Z$ that we had reason to think would be 'close to' its expectation.

--

Now for the expression in the section relating to Satterthwaite in Casella & Berger

Casella & Berger match first and second moments of $Z=\sum_{i=1}^k a_iY_i$ that is, they take $\text{E}(Z) = Z$ and $\text{E}(Z^2)=Z^2$, the second of which gives an estimate of $\nu$.

Note that $Z=\sum_i a_iY_i$ is a constant times a sample expectation; there's a clear sense in which we might expect that $Z\approx \text{E}(Z)$ and $Z^2 \approx \text{E}(Z^2)$, but we don't actually have to justify it here, we're just following their argument about what happens when we do it.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Since we have known the degrees of freedom of $Y_i$s, it is easy to compute exactly the value of $\mathbb{E}[(\sum_{i=1}^ka_iY_i)^2]$, i.e., the population expectation. What is the motivation to use a sample mean here? $\endgroup$ – user72637 May 11 '16 at 5:01
  • $\begingroup$ @NP-hard The issue is that the actual problem is slightly misrepresented in C&B -- in the original problem under discussion (the one being solved by Satterthwaite) we actually don't have chi-square variables, we have sample variances -- which are multiples of chi-square variables with an unknown scale factor because $\sigma_i^2$ is unknown. It must be estimated, so we must use sample variances to do that. So using the method of moments makes sense. If we actually had chi-square variates, it would be a simpler problem. $\endgroup$ – Glen_b May 11 '16 at 10:33
2
$\begingroup$

As pointed out patiently by Glen_b above, since $X_{i}$ is an independent random sample, we have $$E(\sum a_{i}Y_{i})=\sum a_{i}E(Y_{i})=\sum a_{i} Y_{i}$$where the last equation $$E(Y_{i})=Y_{i}$$ follows from the fact that from the definition of method of moments we have $$E(X)=\frac{\sum X_{i}}{n}=\overline{X}=X \text{when $n$=1} $$So the author's proof is justified.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Can you really say $\bar X = X$ at the end there, without say setting $n=1$? Or is that already implied and I missed something? $\endgroup$ – Glen_b Aug 19 '13 at 3:17
  • $\begingroup$ To clarify, it would work if you do that last line differently. $\endgroup$ – Glen_b Aug 19 '13 at 3:41
  • $\begingroup$ @Glen_b: Yeah, I have personal doubts at here. I thought since $X_{i}$ are identically distributed, they all have distribution $X$, so this might work. But after some thought this sounds wrong. $\endgroup$ – Bombyx mori Aug 19 '13 at 16:15
  • $\begingroup$ It's correct as you have it now, but it's perhaps not the most elegant way to do it. $\endgroup$ – Glen_b Aug 19 '13 at 16:59
  • $\begingroup$ @Glen_b: This I surely admit. I was wondering if the last line of derivation enable me to claim $E(Z)=Z$ in general. $\endgroup$ – Bombyx mori Aug 19 '13 at 20:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.