Autocovariance of a non-stationary process

Question

I'm just going to apologize first thing, because I know my understanding of these topics is very lacking.

I'm reading some lecture notes from what appears to be an econometrics course, and they are going over the stationarity of processes. In the course of defining stationarity, they provided the following definition of the autocovariance function:

$$ \gamma(s,t) = Cov(X_s,X_t)$$

They went on to say that for a stationary process, we have the following:

$$ \gamma_X(s,t) = \gamma_X(s+h,t+h) \forall s,t,h,\in \mathbb{Z} $$

and that because of this property, we can rewrite the autocovariance function as

$$ \gamma_X(h) = Cov(X_t, X_t+h) \text{ for } t,h\in\mathbb{Z}$$

I am only familiar with the latter definition of autocovariance. I am confused as to what could be meant by the former, in the case that $\{X_t\}$ is a non-stationary process. Because we're dealing with time series, does it make sense to say "the covariance of $X_t$ and $X_s$?" There will only be one realization of $X$ at time $t$ or $s$, and furthermore only one realization of $X$ that necessarily has the same distribution as $X_t$, so how can we speak of the covariance of $X_s$ and $X_t$?

I'm sorry if this is worded in a confusing way.

Answer 1

The general form $\gamma(s,t)$ refers to the covariance between the value of the series at times $s$ and $t$ when those values are considered as random variables. That is, it is defined by:

$$\gamma(s,t) \equiv \mathbb{E} \Big[ (X_s -\mathbb{E}(X_s))(X_t -\mathbb{E}(X_t)) \Big].$$

In general, the random variables $X_s$ and $X_t$ (for $s \neq t$) can have any joint distribution --- unless it is an assumption of your analysis, you should not assume that they have the same marginal distribution. Regardless, it is possible that these two different random variables are positively or negatively correlated, and the general form of the autocovariance function captures this for any pair of time values. Note also that this covariance refers to the random variables representing the values of the time series at these two points --- once those values are observed they are then treated as constants and are no longer "correlated".

As you correctly note in your question, once you assume that the process is "covariance stationary", this function depends only on the lag $|s-t|$ and so you can reduce the autocovariance function to a univariate function of the lag between the two times. This is a common assumption in time-series analysis, but it does not always hold, so it is useful to start by considering the more general case first.