4
$\begingroup$

Here the $X_i$'s are i.i.d. and such that convergence in distribution for the infinite sum, is guaranteed. Probably the easiest case is when $X_i$ has a Bernouilli($p$) distribution, then $Z$ has a discrete Geometric($1-p$) distribution. I am not interested in that case, but in more complicated examples.

Actually, I am trying to find what kind of distributions $Z$ could have. I am sure the choices are very limited. Those that work are called attractors. I tried a Normal($0, 1$) for $X_i$ and $Z$ has a nice, smooth distribution, but not normal. I tried log-normal too for $X_i$, but in that case there is no convergence. An attractor must satisfy the functional equation $F_Z = F_{X_1(1+Z)}$.

Most surprisingly, I tried $X_i \sim \sin( \pi Y_i)$ with $Y_i$ Normal($0,1$). The resulting distribution for $Z$ is pretty smooth (but not normal), and its variance, based on strong empirical evidence, seems to be exactly equal to $1$. Is this really the case?

I am also interested in the conditions required to have convergence in distribution for the infinite sum.

Update

The reason I failed to obtain convergence with a log-normal $X_i$ is because I picked one with $E(X_i)>1$. More on this to be published in Part II of this problem, here.

$\endgroup$
  • $\begingroup$ is it true that if $X_i$ and $Z$ satisfy $F_Z = F_{X_1(1+Z)}$ then it should satisfy $F_{Z/(1+Z)} = F_{X_1}$? $\endgroup$ – quester Nov 7 at 18:45
  • $\begingroup$ Interesting question, and I am tempted to say yes. Using the notation $Z \sim X_i (1+Z)$ to mean that both sides have the same distribution even though they are different, you can do operations like you would do with the equal sign, that is in this case $Z/(1+Z) \sim X_i$. In short, $\sim$ works just like $=$, except that $\sim$ means the equality is only in distribution. One quick test to do: if $X_i$ is Bernouilli($p$) and $Z/(1+Z) \sim X_i$, do we have $Z$ is geometric($p$)? $\endgroup$ – Vincent Granville Nov 7 at 18:55
  • $\begingroup$ ok supports are different... thanks for answer to my question :) $\endgroup$ – quester Nov 7 at 19:25
6
$\begingroup$

Final update on 11/29/2019: I have worked on this a bit more, and wrote an article summarizing all the main findings. You can read it here.

Surprisingly, there is a simple and general answer to this problem, despite the fact that all the terms in the infinite sum defining $Z$, are correlated. First, let us assume that $|E(X_i)| < 1$. This is required for convergence. Let us also assume that $E(X_i^2)<1$. This guarantees that the variance exists.

We have the following formula for the $k$-th moment, for $k\geq 0$:

$$E(Z^k) = E[(X_i(1+Z))^k]=E(X_i^k)E[(1+Z)^k].$$

It can be re-written as

$$E(Z^k) =\frac{E(X_1^k)}{1-E(X_1^k)} \cdot\sum_{j=0}^{k-1} \frac{k!}{j!(k-j)}E(Z^j).$$

I suspect much simpler recurrence formulas can be found, for $E(Z^k)$. It follows immediately that $E(Z)=E(X_i)/(1-E(X_i))$. Moments of order 2, 3, and so on can be obtained iteratively. A little computation shows that $$Var(Z) = \frac{Var(X_i)}{(1-E(X_i^2))(1-E(X_i))^2}.$$

I checked the formula when $X_i$ is Bernouilli($p$), and it is exactly correct. I also checked empirically when $X_i$ is Uniform$[0,1]$, and it looks correct: $Var(Z) = 0.506$ based on 20,000 simulated $Z$ deviates, while the true value (according to my formula) should be $\frac{1}{2}$.

Now let's look at $X_i = \sin(\pi Y_i)$ with $Y_i \sim$ Normal($0,1$). There is some simplification due to $E(X_i) = 0$ in this case: $Var(Z) = E(X_i^2) / (1 - E(X_i^2))$. To prove that $Var(Z)=1$ amounts to proving that $E(X_i^2) = 1/2$, that is: $$E(X_i^2) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty} x^2 \sin^2(\pi x) e^{-x^2/2} dx = \frac{1}{2} .$$ I computed this integral using WolframAlpha, see here. The approximation is excellent, at least the first 7 digits are correct. However, the exact answer is not $1/2$, but instead $$E(X_i^2) = \frac{1+(4\pi^2-1)\exp(-2\pi^2)}{2} = 0.500000051...$$

Now, to answer the most challenging question - what kind of distribution is an attractor in this framework - we need to look at the formula that gives the moments of $Z$. Clearly, they can be pretty arbitrary, meaning that the class of attractors is very rich. Of course, not all sequences of numbers represent the moments of a distribution. In order to correspond to an actual distribution, moments most satisfy some conditions, see here. A less challenging question is to find a non-trivial distribution that can not be an attractor, that is a distribution that can never be the distribution of the infinite sum $Z$, no matter what the $X_i$'s are. This is the object of the next section.

Distributions that can not be attractors

The distribution for $Z$ is highly constrained. It must satisfy a number of conditions, and thus, few distributions are attractors (though far more than in the central limit theorem framework, where by far the normal distribution is the main attractor, and the only one with a finite variance.) I'll give just one example here, for $Z$ distributions whose support domain is the set of all natural numbers.

Let us consider a very general discrete distribution for $X_i$, with $P(X_i = k) = p_k, k = 0, 1, 2$ and so on. In this case, $Z$'s distribution must also be discrete on the same support domain. This case covers all possible discrete distributions for $Z$, with support domain being the set of natural numbers. Let's use the notation $P(Z=k) = q_k$. Then we have:

  • $P(Z=0) = p_0 = q_0 = P(X_1 =0)$,
  • $P(Z=1) = p_1 p_0 = q_1 = P(X_1 = 1, X_2 =0)$,
  • $P(Z=2) = (p_1^2 + p_2)p_0 = q_2 = P(X_1 = X_2 =1, X_3 =0)+P(X_1 = 2, X_2 =0)$,
  • $P(Z=3) = (p_1^3 + 2 p_1 p_2 + p_3)p_0 = q_3$,
  • $P(Z = 4) = (p_1^4 + 3 p_1^2 p_2 + 2 p_1 p_3+ p_2^2 + p_4 ) p_0 = q_4$.

We don't even need to use the third, fourth or firth equation. Let's focus on the two first ones. The second one implies that $p_1 = q_1 / p_0 = q_1 / q_0$. Thus we must have $q_1 \leq q_0$ for $Z$ to be an attractor. In short any discrete distribution with $P(Z= 0) < P(Z = 1)$ is not an attractor. The geometric distribution is actually an attractor, the most obvious one, and possibly the only one with a simple representation.

Another interesting question is the following: can two different $X_i$ distributions lead to the same attractor? In the case of the central limit theorem, this is true: whether you average exponential, Poisson, Bernoulli or uniform variables, you end up with a Gaussian variable - in this case the universal attractor; exceptions are few (the Lorenz distribution being one of them). The following section provides an answer for a specific attractor.

If $Z$ is the geometric attractor, then $X_i$ must be Bernouilli

Using the same notation as in the previous section, if $Z$ is geometric, then $P(Z = k) = q_k = q_0 (1-q_0)^k$. The equation $p_1 p_0 = q_1 = q_0(1-q_0)$ combined with $p_0 = q_0$ yields $p_1 = 1-q_0$. As a result, $p_0 + p_1 = q_0 + (1-q_0) =1$. Thus if $k> 1$ then $P(X_i = k) = p_k = 0$. This corresponds to a Bernouilli distribution for $X_i$.

Interestingly, the Lorenz attractor in the central limit theorem framework can only be attained if the $X_i$'s themselves have a Lorenz distribution.

Connection with the Fixed-Point theorem for distributions

Consider $Z_k = X_k + X_{k} X_{k+1} + X_{k} X_{k+1} X_{k+2}+ \cdots$. We have $Z_k = X_k (1+ Z_{k+1})$ . As $k\rightarrow \infty, Z_k \rightarrow Z$. The convergence is in distribution. So at the limit, $Z \sim X_i(1+Z)$, that is, the distributions on both sides are identical. Also, $X_k$ is independent of $Z_{k+1}$. In other words, $Z$ (specifically, its distribution) is a fixed-point of the backward stochastic recurrence $Z_k = X_k (1+ Z_{k+1})$. Solving for $Z$ amounts to solving a stochastic integral equation.

$\endgroup$
  • $\begingroup$ No, $Z \neq X_i (1-Z)$. This is true only in distribution, that is, $F_Z = F_{X_i (1-Z)}$. About the second assumption, it is not an issue, there are plenty of similar examples where a solution can be found, the one discussed here is probably the most simple one, based both on theory and empirical evidence: $Z$ has a geometric distribution if and only if $X_i$ has a Bernouilli distribution. $\endgroup$ – Vincent Granville Nov 8 at 23:40
  • $\begingroup$ And yes, this is a problem with plenty of correlated terms, nevertheless, one with a rather simple solution. $\endgroup$ – Vincent Granville Nov 9 at 0:28
  • $\begingroup$ First off, I agree that there is a lot of evidence to support your variance formulation, which leads me to believe I'm just missing something. It reads to me as if you are suggesting that $|E(X_i)| < 1$ and $E(X_i^2) < 1$ implies that $X_i$ and $X_i(1-Z)$ are equal in distribution (or is it $X_i(1+Z)$, there seems to be a discrepancy in the question vs answer). This doesn't make sense to me. Perhaps you are suggesting that if $F_Z$ is an attractor this equality in distribution (and thus in expectation) must hold. This is not intuitive to me, but that would at least explain the first (cont) $\endgroup$ – knrumsey - Reinstate Monica Nov 9 at 1:26
  • $\begingroup$ (cont) equality i asked about in my previous comment. I still don't understand how you justify $E(X_i^k(1-Z)^k) = E(X_i^k)E((1-Z)^k)$, since $X_i$ and $Z$ are clearly not independent. $\endgroup$ – knrumsey - Reinstate Monica Nov 9 at 1:27
  • $\begingroup$ I added a new section at the bottom, see "connection with the fixed-point theorem for distributions". I believe it clarifies my discussion. $\endgroup$ – Vincent Granville Nov 9 at 17:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.