I'm not sure, but I think I've read somewhere that because the Classical Linear Regression model assumes to have a random sample, when researchers they might not be in presence of a sample with that property, they try to use some randomization technique to make sure the usual theory maybe applied.

In Hayashi's Econometrics, chapter 2, he develops the OLS and studies its large-sample properties in a generalization of iid sample. He assumes that the sample is ergodic, and stationary, and that the regressors multiplied by the error terms (the same that define the orthogonality conditions) also follow a martingale difference sequence.

My question is, given a sample, is there a way to know if it satisfies ergodicity, stationarity, and martingale diff. seq.? Also, even if the sample does not satisfy it, is there a way to make sure that we are able to obtain such a sample, when randomization techniques are not possible to apply?

Any help would be appreciated.

P.S.: This question is also posted in Economics Beta, but there I asked for a more applied, experience-based answer.

  • $\begingroup$ In a time series context you assume that your data is stationary and weakly mixing. Stationarity corresponds to the assumption of identical observation while weakly mixing corresponds to independent observations in a cross-sectional setting. For testing stationarity you would use a unit root test, look if the variance is constant over time, if there are structural breaks, if there is a time trend. For independence you would examine the autocor. of the series at hand. The ACF and PACF from the residuals should be white noise which is a form of a mart. diff. seq. I can expand if you want to. $\endgroup$ – Plissken Mar 9 '15 at 10:40
  • $\begingroup$ @Dan I think I'll take up your suggestion. Could you please expand? ;) $\endgroup$ – An old man in the sea. Mar 9 '15 at 18:59
  • $\begingroup$ @Dan, also weakly mixing (ergodic in my case) is an asymptotic independent behaviour. $\endgroup$ – An old man in the sea. Mar 9 '15 at 20:32
  • $\begingroup$ This is a case of someone throws in disparate assumptions so certain results (LLN and CLT) can be quoted, without consideration for consistency. $\endgroup$ – Michael Oct 6 '20 at 19:25

I will demonstrate using an AR(1) model. Consider a variable, $\left(y_{t}\right)_{t=0,1,\ldots} $. Then consider the AR(1) model: $y_{t}=\rho y_{t-1}+\varepsilon_{t},\quad\varepsilon_{t}\thicksim N\left(0,\,\sigma^{2}\right) $.

Solving the AR(1) model we get:$y_{t}=\rho^{t}y_{0}+\sum_{i=0}^{t-1}\rho^{i}\varepsilon_{t-i} $

where $y_{0} $ is the initial value of the process. Finding the variance we get: $\sigma^{2}\frac{1-\rho^{2t}}{1-\rho} $. Clearly this process is not stationary nor weakly mixing (hence its not ergodic since weakly mixing imlies ergodicity) as both the conditional mean and the conditional variance depend upon time. We can therefore write:$y_{t}=\rho^{t}y_{0}+\sum_{i=0}^{t-1}\rho^{i}\varepsilon_{t-i}\stackrel{D}{=}N\left(\rho^{t}y_{0},\,\sigma^{2}\frac{1-\rho^{2t}}{1-\rho^{2}}\right) $

If however the condition $\left|\rho\right|<1 $ holds then the series is stationary and weakly mixing (hence ergodic since weakly mixing implies ergodicity) and we can write it as:$y_{t}=\sum_{i=0}^{t-1}\rho^{i}\varepsilon_{t-i}\stackrel{D}{=}N\left(0,\,\sigma^{2}\frac{1}{1-\rho^{2}}\right) $

since when $\left|\rho\right|<1 $ holds then $\rho^{t}\rightarrow0 $ exponentially fast and we have that $\rho^{t}y_{0}\rightarrow0 $ and $\sigma^{2}\frac{1-\rho^{2t}}{1-\rho^{2}}\rightarrow\sigma^{2}\frac{1}{1-\rho^{2}} $ exponentially fast. To summarize, when $\left|\rho\right|<1$ and $\varepsilon_{t}$ is normally distributed then the condition $\left|\rho\right|<1$ is enough to ensure ergodicity of both the mean and the variance. Now we know that $\left|\rho\right|<1 $ is enough to ensure stationarity and ergodicity of the AR(1) process and in order to test this we could run a unit root test on the series and see if the series had a unit root or not. Other sources of non-stationarity could be a deterministic trend, changing variance and structural breaks, all of which we could model. A unit root however needs a transformation of the data or we could use cointegration analysis if we had more than one series.

To illustrate I will simulate the AR(1) model: $y_{t}=y_{t-1}+\varepsilon_{t} $:

We see that the autocorrelation function decays very slowly and from the time series graph the series does not look very stationary. Therefore we can say that clearly the assumption about stationarity and weakly mixing (hence ergodicity) does not hold when $\left|\rho\right|=1 $. For information about the autocorrelation function see here.

For the second illustration I will simulate the AR(2), $y_{t}=0.3y_{t-1}+0.4y_{t-1}+\varepsilon_{t} $ model but I will try to model it with an AR(1) model to see how the residuals look like.enter image description here

We see that the time series graph looks stationary and the ACF goes exponentially towards zero indicating stationarity and weakly mixing. By fitting an AR(1) model to the data we get the following model: $y_{t}=0.54y_{t-1}+\varepsilon_{t} $. If we now look at the residual ACF we see that they are not white noise (a martingale diff. seq. is white noise) and the portmanteu test rejects the null of no autocorrelation.enter image description here

If we then fit an AR(2) to the simulated data we get an estimated model of: $y_{t}=0.37y_{t-1}+0.32y_{t-1}+\varepsilon_{t} $. Now the residuals do look like white noise and we cannot reject the null of no autocorrelation.enter image description here

If this did not answer your question then let me know.

  • $\begingroup$ Dan, I think your answer is more time-series directed when I was thinking in a cross-section perspective... $\endgroup$ – An old man in the sea. Mar 20 '15 at 17:30
  • $\begingroup$ But you are talking about stationarity, ergodicity and martingale difference sequences which are concepts from time series analysis. It does not make sens to talk about stationarity of cross-sectional data since you only have one time period per observation. And Hayashi's Chap. 2 is concerned with time series. $\endgroup$ – Plissken Mar 20 '15 at 19:50
  • $\begingroup$ Part of chap. 2 is concerned with some time-series related concepts, but if you check how he proves the asymptotic normality of the OLS estimator, he doesn't restrict himself to time-series, or in chapter 3, with the single-equation GMM estimator. In fact, he only speaks of observations, without ever referring to time. At least this is how I understand what I've tried to study. I might be wrong, though... $\endgroup$ – An old man in the sea. Mar 20 '15 at 22:24
  • $\begingroup$ Yes that is true. Try to look at p. 101 where Hayashi defines ergodicity and p. 104 where he defines the martingale difference sequence and you will see that these are time series concepts. Further he says on p. 110 that a special case of ergodic stationarity is i.i.d. which we most often assume when doing cross-sectional regressions: If you assume stationarity and weakly mixing then stationarity will imply identical obs. and weakly mixing will ensure independence. What page number are you referring to? $\endgroup$ – Plissken Mar 21 '15 at 9:58
  • $\begingroup$ I was referring to page 109 and beyond. I thought that maybe ergodic stationarity, could also be referred to cross-section data, as a more general assumption than iid. $\endgroup$ – An old man in the sea. Mar 21 '15 at 12:20

How to detect if Ergodicity, Stationarity and Martingale. dif. sequence?

  • This set of assumptions, taken together, is rather ad hoc and evidently strung together for ease of exposition for a particular audience. One might even argue that they're borderline un-primitive. There is really no compelling reason to test an ad hoc set of assumptions someone threw together.

  • It's pretty clear here that strict stationarity and ergodicity on $(X_i, \epsilon_i)$ are assumed so that a (S)LLN can be quoted, which gives consistency for OLS. Then the MDS condition on $(X_i\epsilon_i)$ is added so that a CLT can be quoted.

  • Martingale and ergodicity/mixing-type of conditions are completely different. Neither one implies the other. (See, for example, this question.) Ergodicity is, by definition, impossible to test statistically---it is an almost-sure condition. There is no statistic to test an $\omega$-by-$\omega$ condition. One should not expect testability to result from adding a completely unrelated condition, such as the martingale property. Same can be said about weak- or strong-mixing conditions. (Weak-mixing is a strengthening of ergodicity under stationariy; strong mixing is a strengthening of weak-mixing.) They are, by definition, impossible to test---there is no statistic that lets you compare independence of measurable sets pairwise.

  • A more primitive, and consistent, set of assumptions would be, for example, that the DGP $(X_i, \epsilon_i)$ is $\alpha$-mixing of sufficiently fast rate with enough number of moments for $X_i$ and $\epsilon_i$, so that both (W)LLN and CLT hold and inference can be carried out.

  • Even then, for reasons given above, such type of assumptions are not meant to be tested. Rather, they are general conditions that you can plausibly argue the DGP falls under.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.