# Law of Large Numbers for geometrically decaying sequences

Let $$(X_n)_n$$ be a sequence of i.i.d. random variables, and let $$\rho \in (0,1)$$. Is there any asymptotic Theorem for the following random variable:

$$Y = \lim_{n\rightarrow \infty}\sum_{i=0}^n \rho^iX_i$$ in the fashion of a Law of large numbers? My conjecture is that $$Y \rightarrow^{a.s.} \frac{\mathbb{E}[X]}{1-\rho}$$, but I am not able to find a proof nor a reference about it.

• The conjecture can't be correct - for example, suppose the $X_i$ are Bernoulli distributed with parameter 0.5 and $\rho$ is 0.1. Then if $X_1 = 0$, it is impossible for $Y$ to ever exceed 0.5, so it can't almost surely be equal to something greater than 0.5. Mar 16, 2021 at 11:25
• That wonderful counterexample by @fblundun exposes the basic problem: as $n$ increases, the contributions of the first few $X_i$ will persist: the only possible way we could hope for a limiting theorem would be for the contributions of the remaining $X_i$ eventually to overwhelm the first few. Since that's impossible when $|\rho|\lt 1,$ pursuing any result of this type looks hopeless.
– whuber
Mar 16, 2021 at 15:34

To see why this is not possible, let $$\psi_Y$$ be the cumulant generating function for $$Y$$. It is defined for any real number $$t$$ by

$$\psi_Y(t) = \log E\left[e^{itY}\right].$$

If the sum in the question converges to a random variable $$Y,$$ then taking $$X$$ to be distributed like all the $$X_i$$ and independent of them we see

$$Y \sim \lim_{n\to\infty} X + \rho X_0 + \rho^2 X_1 + \cdots = X + \lim_{n\to\infty} \rho\left(X_0 + \rho X_1 + \cdots\right) = X + \rho Y .$$

This entails

$$\psi_Y(t)=\psi_X(t) + \psi_Y( \rho t )$$

for all $$t.$$ We may recover $$\psi_X$$ from the distribution $$Y$$ via

$$\psi_X(t) = \psi_Y(t) - \psi_Y(\rho t)$$

and, since the c.g.f $$\psi_X$$ determines the distribution of $$X,$$ we can reverse the process: $$Y$$ determines the common distribution of the $$X_i.$$

Consequently there can be no analog of the laws of large numbers, which posit convergence to universal limiting distributions (under mild assumptions about the $$X_i$$), effectively wiping out almost all information about the underlying distribution used to arrive at the limit.