# Conceptual distinction between heteroscedasticity and non-stationarity

I'm having trouble distinguishing between the concepts of scedasticity and stationarity. As I understand them, heteroscedasticity is differing variabilities in sub-populations and non-stationarity is a changing mean/variance over time.

If this is a correct (albeit simplistic) understanding, is non-stationarity simply a specific case of heteroscedasticity across time?

• Consider the situation where the mean changes over time but the variance does not. – whuber Jan 22 '13 at 23:18

To give precise definitions, let $X_1, \ldots, X_n$ be real valued random variables.

Stationarity is usually only defined if we think of the index of the variables as time. In this case the sequence of random variables is stationary of $X_1, \ldots, X_{n-1}$ has the same distribution as $X_2, \ldots, X_n$. This implies, in particular, that $X_i$ for $i = 1, \ldots, n$ all have the same marginal distribution and thus the same marginal mean and variance (given that they have finite second moment).

The meaning of heteroscedasticity can depend on the context. If the marginal variances of the $X_i$'s change with $i$ (even if the mean is constant) the random variables are called heteroscedastic in the sense of not being homoscedastic.

In regression analysis we usually consider the variance of the response conditionally on the regressors, and we define heteroscedasticity as a non-constant conditional variance.

In time series analysis, where the terminology conditional heteroscedasticity is common, the interest is typically in the variance of $X_k$ conditionally on $X_{k-1}, \ldots, X_1$. If this conditional variance is non-constant we have conditional heteroscedasticity. The ARCH (autoregressive conditional heteroscedasticity) model is the most famous example of a stationary time series model with non-constant conditional variance.

Heteroscedasticity (conditional heteroscedasticity in particular) does not imply non-stationarity in general.

Stationarity is important for a number of reasons. One simple statistical consequence is that the average $$\frac{1}{n} \sum_{i=1}^n f(X_i)$$ is then an unbiased estimator of the expectation $E f(X_1)$ (and assuming ergodicity, which is slightly more than stationarity and often assumed implicitly, the average is a consistent estimator of the expectation for $n \to \infty$).

The importance of heteroscedasticity (or homoscedasticity) is, from a statistical point of view, related to the assessment of statistical uncertainty e.g. the computation of confidence intervals. If computations are carried out under an assumption of homoscedasticity while the data actually shows heteroscedasticity, the resulting confidence intervals can be misleading.

• does stationarity imply homoskedasticity? – adrCoder Dec 2 '20 at 10:29
• @adrCoder It depends on what you mean, precisely, by homoskedasticity. If you mean whether the marginal variances of a time series is constant, the answer is yes. As I write above, a stationary time series with second moment has a constant marginal variance. Often the interest is in the conditional variances, though. A time series can be stationary while having non-constant conditional variances, e.g. ARCH. This can show up in data as clusters of variables with large fluctuations interspersed with periods of small fluctuations. – NRH Dec 2 '20 at 10:49
• When one runs the Augmented Dickey Fuller test on the residuals of a regression, and the Breusch Pagan test, which types does (s)he capture? Is it possible to have stationarity from ADF and heteroskedasticity from Breusch Pagan? I am talking about a small sample here (~40 observations). – adrCoder Dec 2 '20 at 11:03
• @adrCoder I don't see why not, but I couldn't tell for sure with the information given in the comment. Why not ask a new question with the specific data and results? – NRH Dec 2 '20 at 11:46

There are 3 degrees of stationary. The weak form requires mean and variance are kept constant. This means that of 3 stationary definitions are stronger requirements than heteroscedasticity because heteroscedasticity means constant variance, without reference to the mean.

A process can have heteroscedasticity. But if its mean is not constant, then the process is not (weakly) stationary.

A stationary process (let's denote it by 'S') implies homoscedasticity (let's denote it by 'H'). So S --> H.

Naturally its contraposition is also true. So H' --> S', i.e. non-homoscedasticity implies non-stationary.

But the inversion and negation are not true. In other words:

"Non-stationary implies non-homoscedasticity" is not true.

"There exists a stationary process that is non-homoscedasticity" is not true.

A time series is stationary if all its statistical properties do not depend upon the time origin. If this requirement is not fulfilled, the time series is not stationary.

Even a stationary time series cannot be described on the basis of just one sample record. Its statistical properties must be analyzed by averaging over the ensemble of sample records at different time origins.

If statistical properties are the same for any individual sample record and for the case when they are determined through ensemble averaging, the time series is ergodic.

As statistical properties of a heteroscedactic time series are time-dependent, it is not stationary and, of course, not ergodic. Its properties determined for a single sample record cannot be extended to its past and future behavior.

Incidentally, the correlation/regression analysis cannot be applied to time series as dependence between them (the coherence function) is frequency-dependent and can be characterized through (multivariate) stochastic difference equations equations (time domain) or the frequency response function(s) (frequency domain).

Extending regression analysis developed for random variables to time series is erroneous (e.g. see Bendat and Piersol, 2010; Box et al., 2015).