Why is $x_t = x_{t-1}+w_t$ not wide-sense/second-order stationary? I understand that if $x_t$ is $\text{AR}(1)$, i.e.,
$$x_t = \phi x_{t-1}+w_t\tag{1}$$
with $x_t$ (second-order/wide-sense) stationary, $w_t \sim \mathcal{N}(0, \sigma^2_w)$ independent from $x_t$, as long as $\sup_t\text{Var}(x_t) < \infty$ and $|\phi| < 1$, we can write
$$x_t = \sum_{i=0}^{\infty}\phi^{i}w_{t-i}\text{.}$$
I also understand that the autocovariance is given by
$$\gamma(h) = \dfrac{\sigma^2_w \phi^h}{1-\phi^2}\tag{2}$$
as long as $h \geq 0$ and $|\phi| < 1$.
What I don't understand is this:

Suppose $\phi = 1$ in $(1)$. How do we know that $x_t$ isn't
  (wide-sense) stationary?

The sources I've seen online state that this is true because "the variance is infinite" (can't find them right now). This is probably because if $\phi > 0$, we obtain $\lim_{\phi \to 1}\gamma(h) = \infty$ from $(2)$. But, if this is using $(2)$ above, $(2)$ is derived under the assumption of stationarity, which doesn't make any sense.
 A: We need to assume only that


*

*There exists an index $t$ for which $\operatorname{Var}(X_{t})$ is finite.

*$\sigma^2 = \sigma^2_w$ is nonzero.
The independence of $X_{t-1}$ and $w_t$ gives $$\operatorname{Var}(X_t) = \operatorname{Var}(X_{t-1}+w_t)=\operatorname{Var}(X_{t-1}) + \operatorname{Var}(w_t) = \operatorname{Var}(X_{t-1}) + \sigma^2.$$
Comparing the left and right sides in light of $(2)$ shows $\operatorname{Var}(X_{t-1})\ne \operatorname{Var}(X_{t}).$  (Assumption $(1)$ makes this a meaningful statement about finite numbers.)  Consequently $(X_t)$ cannot be stationary--which, among other things, implies every $X_t$ has the same distribution--because it's not even second-order stationary.
A: Now that I've finished a grad-level Time Series course, I can provide an answer. This is essentially exercise 2.8 in Brockwell and Davis' Introduction to Time Series and Forecasting, 3rd edition.
For the $\text{AR}(1)$ model indicated above, we have
$$x_t = \phi x_{t-1} + w_t\tag{*}$$
Using (*) above, for $n$ finite, we have
$$\begin{align}
x_t &= \phi x_{t-1} + w_t \\
&= \phi(\phi x_{t-2}+w_{t-1})+w_t \\
&= \phi^2 x_{t-2}+\phi w_{t-1}+w_t \\
&= \phi^2(\phi x_{t-3}+w_{t-2})+\phi w_{t-1}+w_t \\
&= \phi^3x_{t-3}+\phi^2w_{t-2}+\phi w_{t-1} + w_t \\
&\vdots \\
&= \phi^{n+1}x_{t-(n+1)}+\sum_{k=0}^{n}\phi^{k}w_{t-k}\text{.}
\end{align}$$
This yields the equation
$$x_t - \phi^{n+1}x_{t-(n+1)}  = \sum_{k=0}^{n}\phi^kw_{t-k}\text{.}$$
With $w_t \overset{\text{iid}}{\sim}\mathcal{N}(0, \sigma^2_w)$, then
$$\text{Var}\left(\sum_{k=0}^{n}\phi^kw_{t-k} \right) = \sum_{k=0}^{n}\phi^{2k}\sigma^2_w=(n+1)\sigma^2_w$$
with the assumption that $\phi = 1$. 
Assume by contradiction that the $x_t$ are stationary. Because the $x_t$ are stationary, the covariances $\text{Cov}(x_t, x_{t+h})$ are independent of $t$; in particular, $\text{Cov}(x_t, x_{t}) = \text{Var}(x_t)$ is independent of $t$. Write $\sigma^2_{X} = \text{Var}(x_t)$. Then
$$\begin{align}
\text{Var}(x_t - \phi^{n+1}x_{t-(n+1)}) &= \sigma^2_X + \phi^{2(n+1)}\sigma^2_X +2\cdot \text{Cov}(x_t, - \phi^{n+1}x_{t-(n+1)}) \\
&= \sigma^2_X + \phi^{2(n+1)}\sigma^2_X  -2\phi^{n+1}\cdot\text{Cov}(x_t, x_{t-(n+1)}) \\
&\leq \sigma^2_X[1+\phi^{2(n+1)}] \tag{**} \\
&= 2\sigma^2_X
\end{align}$$
hence for any $n$,
$$(n+1)\sigma^2_w \leq 2\sigma^2_X\text{.}$$
Taking the limit as $n \to \infty$, it follows that $\sigma^2_X > \infty$, hence contradicting that $\sup_t \text{Var}(x_t) < \infty$ by stationarity.
In (**), I made an assumption that $\text{Cov}(x_t, x_{t-(n+1)}) \geq 0$. To see why this is the case, write
$$\begin{align}
\text{Cov}(x_t, x_{t-(n+1)}) &= \text{Cov}\left(x_t, \dfrac{x_t - \sum_{k=0}^{n}\phi^k w_{t-k}}{\phi^{n+1}}\right) \\
&= \dfrac{1}{\phi^{n+1}}\text{Cov}\left(x_t, x_t-w_t \right) \\
&= \dfrac{\sigma^2_X}{\phi^{n+1}} \\
&\geq 0
\end{align}$$
due to independence of the $w_t$ from the $x_t$.
