7
$\begingroup$

I'm trying to work through the proofs of the asymptotic distributions of OLS. In this derivation, we make the assumptions that the observations $(y_n,\mathbf{x}_n)$ are i.i.d. and that the fourth moments exist. The book I'm using (Ruud) says that the second-order moments of the random variables $x_{nk}x_{nj}$, $x_{nk}y_n$, and $y_n^2$ are functions of the fourth-order moments of $(y_n,\mathbf{x}_n)$. Is this supposed to be obvious? Why is it the case that the second moments are functions of the fourth moments?

$\endgroup$
3
  • $\begingroup$ The heart of the matter is that $2\times 2=4$ and twice any nonnegative integer less than $2$ is less than $4$. $\endgroup$
    – whuber
    May 18, 2017 at 18:54
  • $\begingroup$ Am I just dense? For example, the variance of $x_{nk}x_{nj}$ is a second order central moment. So $Var(x_{nk}x_{nj}) = E(x_{nk}^2x_{nj}^2) - E(x_{nk}x_{nj})^2$, which is a function of a fourth order moment and a second order moment. Is it as simple as that? $\endgroup$
    – user21359
    May 18, 2017 at 19:17
  • $\begingroup$ Yes: you got it. $\endgroup$
    – whuber
    May 18, 2017 at 19:58

1 Answer 1

5
$\begingroup$

This comes up a lot in various contexts and therefore deserves a general answer. The question concerns pairs of random variables, so let's simply call them $X$ and $Y$. Think of them as the components of a bivariate random variable $(X,Y)$ with some distribution $F$. The properties of $F$ won't matter: the following exposition is completely general. It assumes only that any expectation that is taken actually exists and is finite.

A (raw) moment is given by powers $r$ and $s$, usually taken to be small nonnegative integers $0,1,2,\ldots$. By definition it is

$$\mu_F(r,s) = E_F(X^rY^s).\tag{1}$$

You see, it's merely the expectation of a monomial function of $(X,Y)$. When $s=0$, $Y^s=1$ disappears and we are left with the usual (univariate) moments,

$$\mu_F(r,0) = \mu_F(r) = E_F(X^r).$$

Let's take a little tangent to see that central moments don't introduce any real complication.

The central moments are linear combinations of the raw moments with universal coefficients (they don't depend on $F$, only on $r$ and $s$). By definition they are

$$\mu^\prime_F(r,s) = E_F((X-\bar X)^r(Y-\bar Y)^s).\tag{2}$$

(As is usual, I have written $\mu_F(1,0) = \bar X$ for the expectation of $X$ and $\mu_F(0,1)=\bar Y$ for the expectation of $Y$.) Ordinary algebra (the Binomial Theorem) lets us expand the right hand side of $(2)$:

$$(X-\bar X)^r(Y-\bar Y)^s = \sum_{i=0}^r\sum_{j=0}^s \binom{r}{i}\binom{s}{j}X^i Y^j \bar X^{r-i} \bar Y^{s-j}.$$

Note the Binomial coefficients $\binom{r}{i}$ and $\binom{s}{j}$, which do not depend on $F$. Linearity of expectation says if we take the expectation of the left hand side, it will be the combination of expectations on the right. Bear in mind that since $\bar X$ and $\bar Y$ are numbers, they introduce no complications:

$$\eqalign{\mu^\prime_F(r,s)&=E((X-\bar X)^r(Y-\bar Y)^s) \\&= \sum_{i=0,j=0}^{r,s} \binom{r}{i}\binom{s}{j}E(X^i Y^j)\bar X^{r-i} \bar Y^{s-j} \\&= \sum_{i=0,j=0}^{r,s} \binom{r}{i}\binom{s}{j}\mu_F(i,j)\mu_F(1,0)^{r-i}\mu_F(0,1)^{s-j}.} $$

This exhibits the central moments $\mu^\prime$ as linear combinations of the raw moments $\mu$.

Returning to the question, consider how you would go about computing, say, the variance of any monomial $X^rY^s$. By one definition of variance (as the expected square minus the square of the expectation) and the definition of moments $(1)$, this is

$$\operatorname{Var}(X^rY^s) = E((X^rY^s)^2) - E(X^rY^s)^2 = \mu_F(2r,2s) - \mu_F(r,s)^2.$$

Nothing new appears: moments of powers of the variables are linear combinations of moments of the original variables.

It's that simple, and the generalizations to higher moments (central or not) of monomials, and to more than two variables, ought now to be obvious.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.