The $X_i$'s are i.i.d. and $X$ denotes any of these random variables. We assume here that $|E(X)|<1$ to guarantee convergence. I am interested in particular in the third moment $E(Y^3)$. For the first two moments, we have (see here):

$$E(Y) = \frac{E(X)}{1-E(X)},\mbox{ Var}(Y)=\frac{\mbox{Var}(X)}{(1-E^2(X))(1-E(X^2))}.$$

The reason for my interest is as follows. Let $$Z = X_1 + X_1 X_2 + X_1 X_2 X_3 + \cdots .$$

If $E(X)=0$, then $E(Y) = E(Z)$ and $E(Y^2)=E(Z^2)$, see here. My hope is that this is no longer true for higher moments, that is, $E(Y^3) \neq E(Z^3)$. Ultimately, that's what I'd like to prove. While the two first moments are not enough to make the model identifiable ($Y$ vs. $Z$) it is my hope that by using the first three moments as model parameters, this is enough to discriminate between $Y$ and $Z$, and thus make the model identifiable. Note that $$E(Z^3) =\frac{E(X^3)(1+3E(Z)+3E(Z^2))}{1-E(X^3)}.$$ The result for $E(Z^3)$ comes from section 4.2. in this article.


A possible way to compute $E(Y^n)$ is as follows. Define $Y_k$ as the sum of the first $k$ terms in $Y=X_1 + X_2 X_3 + X_4 X_5 X_6 +\cdots$. Then $Y_{k+1}=Y_k + V_{k+1}$ with $V_{k+1}$ being a product of $k+1$ i.i.d. random variables with same distribution as $X$. Also, $Y_k$ and $V_{k+1}$ are independent. Thus

$$E(Y_{k+1}^n) =\sum_{i=0}^n \frac{n!}{i!(n-i)!}E(Y_k^i) (E(X^{n-i}))^{k+1}.$$

We can focus on the case $n=3$ to begin with. The above recurrence relation (if correct) could lead to a solution. We are interested in the case $k\rightarrow\infty$, as $Y_k \rightarrow Y$ (in distribution.) We also have the following recursion:

$$E(Y_{k+1}^n) =E(Y_k \cdot Y_{k+1}^{n-1}) + (E(X))^{k+1}\cdot E(Y_{k+1}^{n-1}).$$

Update 2

If $X$ has the distribution $P(X=-0.5) = 0.5, P(X=0.5) = 0.5$ then both $Y, Z$ have the same uniform distribution on $[-1, 1]$. It is then impossible to discriminate between models $Y$ or $Z$.

I also looked at the case $X$ = Normal$(0, 1/4)$. The variances for $Y$ or $Z$ are both identical as expected, confirmed by empirical evidence, and both are equal to $1/3$ as expected. Yet higher moments are different. Below is the chart showing the empirical percentile distribution, comparing $Y$ (blue) vs. $Z$ (red), if $X$ is Normal$(0, 1/4)$. They are clearly different.

enter image description here

  • $\begingroup$ This is cool. I'm just curious: why are you doing it? $\endgroup$ – eric_kernfeld Dec 3 '19 at 18:41
  • $\begingroup$ I am interested in parameter estimation for the models $Y$ and $Z$, being able to discriminate between $Y$ and $Z$, and trying to uncover the distribution of the hidden layer ($X$) if you only observe $Y$ or $Z$. More generally, here is what I am really interested in (long article): datasciencecentral.com/profiles/blogs/… $\endgroup$ – Vincent Granville Dec 3 '19 at 18:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.