Why Does Second Order Weak Stationarity include Statement on Covariances in addition to Statement on Mean and Variance? A stochastic process is second order weakly stationary if all random variables have same mean (first moment), and same variance (second moment?), and covariances that are time-invariant (second moment as well?). I expected this definition to contain a statement about the mean (first moment) and the variance (second moment). How does covariance enter? Is covariance a moment? When is variance the second moment and and when is it an incomplete description of a second moment? Please clarify. 
 A: The covariances are part of the second-order moment: In time-series problems you are dealing with random vectors indexed by time.  When you have a random vector $\mathbf{X} = (X_1,...,X_n)$ its first and second-order moments are the mean vector and covariance matrix:
$$\mathbb{E}(\mathbf{X}) = \begin{bmatrix} \mathbb{E}(X_1) \\ \mathbb{E}(X_2) \\ \vdots \\ \mathbb{E}(X_n) \end{bmatrix} \quad \quad \quad
\mathbb{V}(\mathbf{X}) = \begin{bmatrix} \mathbb{V}(X_1) & \mathbb{C}(X_1, X_2) & \cdots & \mathbb{C}(X_1, X_n) \\ \mathbb{C}(X_2, X_1) & \mathbb{V}(X_2) & \cdots & \mathbb{C}(X_2, X_n) \\ \vdots & \vdots & \ddots & \vdots \\ \mathbb{C}(X_n, X_1) & \mathbb{C}(X_n, X_2) & \cdots & \mathbb{V}(X_n) \end{bmatrix}.$$
As you can see, the variance matrix is composed of the variances of the individual elements of the random vector and also their covariances.  Moreover, the scalar variance values are just covariances of the elements with themselves.  Second-order stationarity imposes a condition on these moments that they are affected by the time indices only through the time-lag between values.  
