9
$\begingroup$

The $X_i$'s are i.i.d. and $X$ denotes any of these random variables. We assume here that $|E(X)|<1$ to guarantee convergence. I am interested in particular in the third moment $E(Y^3)$. For the first two moments, we have (see here):

$$E(Y) = \frac{E(X)}{1-E(X)},\mbox{ Var}(Y)=\frac{\mbox{Var}(X)}{(1-E^2(X))(1-E(X^2))}.$$

The reason for my interest is as follows. Let $$Z = X_1 + X_1 X_2 + X_1 X_2 X_3 + \cdots .$$

If $E(X)=0$, then $E(Y) = E(Z)$ and $E(Y^2)=E(Z^2)$, see here. My hope is that this is no longer true for higher moments, that is, $E(Y^3) \neq E(Z^3)$. Ultimately, that's what I'd like to prove. While the two first moments are not enough to make the model identifiable ($Y$ vs. $Z$) it is my hope that by using the first three moments as model parameters, this is enough to discriminate between $Y$ and $Z$, and thus make the model identifiable. Note that $$E(Z^3) =\frac{E(X^3)(1+3E(Z)+3E(Z^2))}{1-E(X^3)}.$$ The result for $E(Z^3)$ comes from section 4.2. in this article.

Update

A possible way to compute $E(Y^n)$ is as follows. Define $Y_k$ as the sum of the first $k$ terms in $Y=X_1 + X_2 X_3 + X_4 X_5 X_6 +\cdots$. Then $Y_{k+1}=Y_k + V_{k+1}$ with $V_{k+1}$ being a product of $k+1$ i.i.d. random variables with same distribution as $X$. Also, $Y_k$ and $V_{k+1}$ are independent. Thus

$$E(Y_{k+1}^n) =\sum_{i=0}^n \frac{n!}{i!(n-i)!}E(Y_k^i) (E(X^{n-i}))^{k+1}.$$

We can focus on the case $n=3$ to begin with. The above recurrence relation (if correct) could lead to a solution. We are interested in the case $k\rightarrow\infty$, as $Y_k \rightarrow Y$ (in distribution.) We also have the following recursion:

$$E(Y_{k+1}^n) =E(Y_k \cdot Y_{k+1}^{n-1}) + (E(X))^{k+1}\cdot E(Y_{k+1}^{n-1}).$$

Update 2

If $X$ has the distribution $P(X=-0.5) = 0.5, P(X=0.5) = 0.5$ then both $Y, Z$ have the same uniform distribution on $[-1, 1]$. It is then impossible to discriminate between models $Y$ or $Z$.

I also looked at the case $X$ = Normal$(0, 1/4)$. The variances for $Y$ or $Z$ are both identical as expected, confirmed by empirical evidence, and both are equal to $1/3$ as expected. Yet higher moments are different. Below is the chart showing the empirical percentile distribution, comparing $Y$ (blue) vs. $Z$ (red), if $X$ is Normal$(0, 1/4)$. They are clearly different.

enter image description here

$\endgroup$
3
  • $\begingroup$ This is cool. I'm just curious: why are you doing it? $\endgroup$ Dec 3, 2019 at 18:41
  • $\begingroup$ I am interested in parameter estimation for the models $Y$ and $Z$, being able to discriminate between $Y$ and $Z$, and trying to uncover the distribution of the hidden layer ($X$) if you only observe $Y$ or $Z$. More generally, here is what I am really interested in (long article): datasciencecentral.com/profiles/blogs/… $\endgroup$ Dec 3, 2019 at 18:49
  • $\begingroup$ I followed the reference about $E(Z^3)$, but in that sec 4.2 I see a restatement of the formula -- where do I find the proof? $\endgroup$
    – Matt F.
    Jul 28, 2021 at 19:42

1 Answer 1

3
$\begingroup$

Let $t_1=X_1$ and $t_2=X_2X_3$, etc., so $Y=\sum_i t_i$. Then with some help from Mathematica (like this):

\begin{align} E[Y^3] &= E\left[6\sum_{i<j<k}t_it_jt_k + 3\sum_{i\neq j}t_i^2 t_j + \sum_{i}t_i^3\right] \\ &= 6\sum_{i<j<k}E[X]^{i+j+k} + 3\sum_{i\neq j}E[X^2]^iE[X]^j + \sum_{i}E[X^3]^i \\ &= \frac{6E[X]^6}{(1-E[X])(1-E[X]^2)(1-E[X]^3)}\\ \\[-10pt] &\ \ + \frac{3E[X]E[X^2]}{(1-E[X])(1-E[X^2])} - \frac{3E[X]E[X^2]}{1-E[X]E[X^2]} + \frac{E[X^3]}{1-E[X^3]} \\ \end{align}

For $Z$, we can similarly let $u_1=X_1$ and $u_2=X_1X_2$, etc., so $Z=\sum_i u_i$. Then:

\begin{align} E[Z^3] &= E\left[6\sum_{i<j<k}u_iu_ju_k + 3\sum_{i\neq j}u_i^2 u_j + \sum_{i}u_i^3\right] \\ &= 6\sum_{i<j<k}E[X^3]^iE[X^2]^{j-i}E[X]^{k-j} + 3\sum_{i< j}E[X^3]^iE[X]^{j-i} \\ &\ \ + 3\sum_{i>j}E[X^3]^jE[X^2]^{i-j}+ \sum_{i}E[X^3]^i \\ &= s_3(6s_2s_1+ 3s_1 + 3s_2+ 1) \end{align} where $s_n=\sum_i E[X^n]^i=E[X^n]/(1-E[X^n])$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.