Writing AR(1) as a MA($\infty$) process The AR(1) process is

$$
X_t = \phi X_{t-1} + \varepsilon_t
$$

if we use this formula recursively, we get
$$
X_t = \phi(\phi X_{t-2} + \varepsilon_{t-1}) + \varepsilon_t = \phi^2X_{t-2} + \phi\varepsilon_{t-1} + \varepsilon_t = \cdots = \phi^k X_{t-k} + \sum_{j=0}^k \phi^j\varepsilon_{t-j}
$$
If we let $k\to\infty$, we get
$$
X_t = \lim_{k\to\infty}(\phi^k X_{t-k} + \sum_{j=0}^k \phi^j\varepsilon_{t-j}) = \lim_{k\to\infty}(\phi^k X_{t-k}) + \sum_{j=0}^\infty \phi^j\varepsilon_{t-j}
$$
The duality between AR(1) and MA($\infty$) states that there is an equivalence between the two, and that we can write $X_t$ as 

$$
X_t = \sum_{j=0}^\infty \phi^j\varepsilon_{t-j}
$$

The difference between the two results is the term $\lim_{k\to\infty}(\phi^k X_{t-k})$, which should be zero, but how do I show this?
Assuming $|\phi| < 1$, we have that $\lim_{k\to\infty}\phi^k = 0$ of course, but I don't see why $\lim_{k\to\infty} X_{t-k} < \infty$? Does convergence asuume the law of large numbers, or is there another way to show equivalence?

I know there is a proof which inverts the lag operator $1-B$, but I didn't find any justification for why the operator can even be inverted, so I wanted an alternative proof, as the one above.
 A: The usual sense in which convergence is understood in this case is in mean square:
$$
E[Y_t-(\epsilon_t+\phi\epsilon_{t-1}+\phi^2\epsilon_{t-2} +\ldots+\phi^j\epsilon_{t-j})]^2=\phi^{2(j+1)} E[Y_{t-j-1}]^2
$$
If $Y_t$ is stationary
$$
E[Y_{t-j-1}]^2=\gamma_0+\mu^2
$$
Hence
$$
\lim_{j\to\infty}E[Y_t-(\epsilon_t+\phi\epsilon_{t-1}+\phi^2\epsilon_{t-2} +\ldots+\phi^j\epsilon_{t-j})]^2=0
$$
A: You are right to be suspicious of this step, and in fact, without further assumptions to limit the size of $X_{-\infty}$ you cannot get the required form.  Remember that the recursive equation for the AR model is insufficient to yield the joint distribution of the process.  (You need to impose a distribution for the error process, and even then, you either need to impose stationarity or specify an initial distribution that leads to some non-stationary model.)  If you only have this recursive equation, there is no reason that the time-series values could not explode out to large values as $t \rightarrow -\infty$.
For example, the deterministic non-stationary AR process $X_t = \phi^t$ satisfies the recursive equation you have specified (with zero errors), and in this case you have $\lim_{k \rightarrow \infty} X_{t-k} = \infty$.  In this model, for any $\phi \neq 0$ you also have:
$$\phi^k X_{t-k} = \phi^k \phi^{t-k} = \phi^t \neq 0.$$
Since the errors are zero in this deterministic model, this gives you the limiting result:
$$X_t = \underbrace{\sum_{k=0}^\infty \phi^k \varepsilon_{t-k}}_{0} + \underbrace{\lim_{k \rightarrow \infty} \phi^k X_{t-k}}_{\phi^t}. \\[6pt]$$
Clearly, in this case, the limiting term is non-zero, and cannot be removed from the result.  If you would like to be able to remove this last term, you need to add further assumptions to your model (e.g, stationarity).
