A stochastic process is second order weakly stationary if all random variables have same mean (first moment), and same variance (second moment?), and covariances that are time-invariant (second moment as well?). I expected this definition to contain a statement about the mean (first moment) and the variance (second moment). How does covariance enter? Is covariance a moment? When is variance the second moment and and when is it an incomplete description of a second moment? Please clarify.

  • 2
    $\begingroup$ You are looking at defective definitions of what is meant by weak stationarity (also called wide-sense stationarity in the literature). See my answer to this question and an answer of mine on dsp.SE that is referenced therein. $\endgroup$ – Dilip Sarwate Nov 29 '18 at 20:16
  • $\begingroup$ Thank you, Dilip. I looked at the referenced response. What I am not clear about is the relationship between "wide-sense-stationarity",(which I understand to be what others call "covariance stationarity" or "weak stationarity") and "stationary to order 2". What else in addition to "stationary to order 2" is assumed to get to "wide-sense-stationarity"? My question above is meant to be about covariance stationarity. $\endgroup$ – ColorStatistics Nov 29 '18 at 20:42
  • 1
    $\begingroup$ See this answer of mine also over on dsp.SE for an example of a weakly stationary process that is not even stationary to order 1 let alone order 2. Weak stationarity does say something specific about pairs of random variables separated in time by a fixed amount of time $\tau$: all such pairs of random variables must have the same covariance. (Pairs separated by a different amount of time will typically have a different covariance). A special case for $\tau=0$, covariance = variance and so all the random variables have the same variance. $\endgroup$ – Dilip Sarwate Nov 29 '18 at 21:12
  • $\begingroup$ In short, your question "how does covariance enter?" makes no sense if you use the generally accepted definition of weak stationarity: the covariance is right there as part of the definition of weak stationarity. $\endgroup$ – Dilip Sarwate Nov 29 '18 at 21:14
  • 1
    $\begingroup$ Yes, you are correct. $\endgroup$ – Dilip Sarwate Nov 30 '18 at 12:30

The covariances are part of the second-order moment: In time-series problems you are dealing with random vectors indexed by time. When you have a random vector $\mathbf{X} = (X_1,...,X_n)$ its first and second-order moments are the mean vector and covariance matrix:

$$\mathbb{E}(\mathbf{X}) = \begin{bmatrix} \mathbb{E}(X_1) \\ \mathbb{E}(X_2) \\ \vdots \\ \mathbb{E}(X_n) \end{bmatrix} \quad \quad \quad \mathbb{V}(\mathbf{X}) = \begin{bmatrix} \mathbb{V}(X_1) & \mathbb{C}(X_1, X_2) & \cdots & \mathbb{C}(X_1, X_n) \\ \mathbb{C}(X_2, X_1) & \mathbb{V}(X_2) & \cdots & \mathbb{C}(X_2, X_n) \\ \vdots & \vdots & \ddots & \vdots \\ \mathbb{C}(X_n, X_1) & \mathbb{C}(X_n, X_2) & \cdots & \mathbb{V}(X_n) \end{bmatrix}.$$

As you can see, the variance matrix is composed of the variances of the individual elements of the random vector and also their covariances. Moreover, the scalar variance values are just covariances of the elements with themselves. Second-order stationarity imposes a condition on these moments that they are affected by the time indices only through the time-lag between values.

| cite | improve this answer | |
  • $\begingroup$ If weak stationarity/second-order stationarity only imposes that the first moment E(X) and second moment V(X) do not depend on time t, then am I to conclude that weak stationarity says nothing about all the elements of E(X) being identical [i.e. E(X1)=E(X2)=...=E(Xn)], because couldn't all these elements not depend on time and yet be unequal. And similarly am I to conclude that weak stationarity says nothing about the diagonal elements of V(X), the variances, being identical [i.e. V(X1)=V(X2)=...=V(Xn)], because couldn't they not depend on time and yet be unequal? $\endgroup$ – ColorStatistics Nov 30 '18 at 19:12
  • 1
    $\begingroup$ @ColorStatistics: The index is what we mean by "time", so stationarity would mean that $\mathbb{E}(X_t)$ does not depend on $t$. That immediately gives you $\mathbb{E}(X_1) = \cdots = \mathbb{E}(X_n)$ and $\mathbb{V}(X_1) = \cdots = \mathbb{V}(X_n)$. The covariances may depend on time only through the time-lag (i.e., index lag) between elements. $\endgroup$ – Ben Nov 30 '18 at 22:23
  • $\begingroup$ Thank you for this clarification, Ben. It all makes sense now. $\endgroup$ – ColorStatistics Nov 30 '18 at 22:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.