# Why is weak stationarity equivalent to strict stationarity only when distribution is normal?

I read that this is because the mean and variance completely specify a normal distribution. However, e.g. for a Poisson distribution the mean, variance and third moment can all be expressed with the same parameter lambda. Is this not what it means to 'completely specify' a distribution?

What is an example of two Poisson distributions having the same mean, variance and different higher moments? If we treated weak stationary as if it was strict stationarity, what would happen when we try to do time series analysis?

• The "distribution" in a stochastic process $(y_t)$ refers to the multivariate distribution, not the collection of univariate distributions: you are forgetting about the higher-order "correlations" between the variables, such as $\mathbb{E}(y_ty_sy_r)$ for $t \gt s \gt r$. – whuber May 31 '17 at 1:37

The hard part of the question is the second part about exhibiting a non-stationary, weakly stationary series with identical distributions at all times.

I hope you won't mind a simplification: rather than using Poisson distributions, which can take on infinitely many values, let's use one that takes on only a small number of values. A particularly simple distribution $F$ describes the difference between the number of heads and number of tails in two independent throws of a fair coin. (This makes it a version of a Binomial distribution.) It therefore has a chance of $1/2$ of being $0$ and a $1/4$ chance each of being $-2$ and $2$. You can readily compute that its expectation is $0$ and its variance is $2$.

I need to describe two different bivariate distributions, $G$ and $H$, whose marginals both equal $F$.

• $G$ is the joint distribution of two independent random variables $X,Y$ distributed according to $F$. It assigns positive probabilities to all nine distinct outcomes $(X,Y)$: when both of $|X|$ and $|Y|$ are $2$, the chance is $1/16$; the chance of $(0,0)$ is $1/4$; and the other four chances are $1/8$.

• $H$ assigns positive probabilities only to five of these outcomes: when both of $|X|$ and $|Y|$ are $2$, the chance is $1/8$, and the chance of $(0,0)$ is $1/2$.

(One way to realize $H$ in a simulation is to draw $X$ from $F$, and then negate $X$ with probability $1/2$ to produce $Y$.)

It is immediate that the covariance of $X$ and $Y$ under the law $G$ is zero, because $X$ and $Y$ are independent. Under the law $H$, the covariance also is zero, because (since the expectations are zero), $$\operatorname{Cov}_H(X,Y)=\mathbb{E}_H(XY) - \mathbb{E}_H(X)\mathbb{E}_H(Y) = \mathbb{E}_H(XY)$$

and a direct calculation from the definition of expectation gives $$\mathbb{E}_H(XY) = \frac{1}{8}(2\times 2 + 2\times (-2) + (-2)\times 2 + (-2)\times (-2)) + \frac{1}{2}(0\times 0) = 0.$$

We have established that $G$ and $H$ have the same first and second bivariate moments and identical marginal distributions. They provide a nice example of a weakly stationary time series $(Z_t),\ t\in\mathbb{Z},$ that is not stationary. Simply let $(X_t),\ t\in\mathbb{Z}\setminus\{0,1\}$ be a sequence of independent random variables with distribution $F$ and let $(X_0,X_1)$ be independent of the other $X_t$ and have joint distribution $H$. Thus, all bivariate distributions of $(X_t, X_s)$ follow the law $G$ except when $\{t,s\}=\{0,1\}$, which follow the law $H$. Since $H$ differs from $G$, this makes the time series non-stationary. (The illustrations at the end help you see this.)

Nevertheless, all the univariate distributions are the same and all the covariances between $X_t$ and $X_s$ for $t\ne s$ are zero, as we have seen. That makes the series weakly stationary.

One way to illustrate the situation is to show some partial realizations of this process. What is particularly revealing are the first differences of the time series, $dX_t = X_{t+1} - X_t$. These will be values in the set $\{-4,-2,0,2,4\}$. The construction of $H$ precludes the values $\pm 2$ from ever occurring for $dX_0 = X_1 - X_0$, but those values can occur for all other differences. In the graphics below I have therefore plotted these first differences against time $t=-15, -14, \ldots, 15$. The value $dX_0$ is highlighted in red. The "odd" values $\pm 2$ are plotted with crosses and the other values with dots.

It helps to see many realizations at once. In the next graphic I overlaid 300 of them on the same axes.

The pattern becomes clear: at time $t=0$, there are never any differences of $2$ or $-2$ plotted, but such differences show up at all other times. That's a visible demonstration of non-stationarity of the differences. Since the differences are not stationary, the original series cannot be either.

The R code to produce these simulations is particularly simple, so it might be useful to see it.

times <- (-15):16
n <- length(times)
n.reps <- 300
realizations <- replicate(n.reps, {
x <- (rbinom(n, 2, 1/2)-1)*2        # Draw independent Binomial variates
names(x) <- times
x["1"] <- sample(c(-1,1), 1)*x["0"] # Create a dependence at times 0 and 1
x
})

• This really clears things up. I just joined so can't accept answers, sorry! – JTicker Jun 5 '17 at 3:29
• Thank you for your comment. Actually, you can accept answers and upvote them if you wish. Both actions should get you a few reputation points as well, enabling you to do more on the site. – whuber Jun 5 '17 at 13:24

The question that you ask "What is an example of two Poisson distributions with the same mean, variance and different higher order moments" has the answer that there are no such distributions. You are seeking a validation of the false assertion that weak stationarity of a normal process implies strong stationarity of the process because the distribution of a normal random variable depends only on the mean and the variance, and both these have the same values for all the random variables that comprise a weakly stationary random process. Ergo, a weakly stationary normal process is also strongly stationary. As whuber's comment points out, it is necessary to consider the multivariate distributions of the process, not just the univariate distributions of the process.

Generally, weak stationarity does not imply strict stationarity. An example of such a process is given in whuber's answer which constructs a weakly stationary discrete-time process in which all the first-order distributions are the same. An example of a continuous-time weakly stationary process (for which the first-order distributions are not the same) can be found in my answer to the question "If the mean of a random process is constant, does it imply the process is first order stationary?" on dsp.SE.