Stationary function

I am reading Karl Rasmussen's book on Gaussian processes and in the introductory chapter he has the following statement:

Note that the functions in Figure 1.1(a) are smooth and stationary (informally,
stationarity means that the functions look similar at all x locations)


I really do not understand this intuition behind stationary functions. Can anyone explain what he is saying here? What I am confused is by the phrase "functions look similar at all x locations". Does that not mean that the function is simply a constant function with the same output for every x, which is not what he shows in that figure.

• These functions in the figure are random or not? – mpiktas Oct 15 '14 at 13:30
• They seem random. There is some 0 mean prior (does not say anything about the covariance) and the functions are sampled from this prior distribution. I have attached the figure. – Luca Oct 15 '14 at 13:34
• The last part of this very long-winded answer over on dsp.SE has an explanation of stationarity in the context of Gaussian processes. For a stationary Gaussian process, the value of a sample function at any time $t$ can be regarded as a sample of a $N(\mu,\sigma^2)$ random variable: same $\mu$, same $\sigma^2$ for all $t$, and samples at $t_1$ and $t_2$ can be regarded as samples of a bivariate (a.k.a jointly) Gaussian distribution with the covariance depending on the difference $|t_1-t_2|$ and not on the individual values of $t_1$ and $t_2$. – Dilip Sarwate Oct 15 '14 at 13:47
• You may be interested in this post. Basically it states that in a stationarity process the mean, variance, and higher order moments are the same regardless of the time point at which the process is observed. That is, the mean, variance,... remain constant throughout the sample. – javlacalle Oct 15 '14 at 13:56

1 Answer

Stationarity is a concept defined for stochastic processes. Since we look at the process as random function then we can extend the definition of stationarity to functions. I.e a stationary function is a realisation of stationary process. This extension is quite informal though.

Stationarity for a process means that the for a number of points $t_1,...,t_n$, $(X_{t_1},...,X_{t_n})\sim (X_{t_1+h},...,X_{t_n+h})$, where $\sim$ means equality in distribution. The question would be, what does this entail? For example take two points of the process which are close to each other $(X_t,X_{s})$. The relationship between these two points is expressed by their distribution function. Let us say the points are closely related, i.e. the correlation $corr(X_t,X_s)=0.9$. Then if we shift these points by any distance, due to stationarity their relationship remains the same, i.e. $corr(X_t,X_s)=corr(X_{t+h},X_{s+h})=0.9$. Informaly in this case we can say that $X_{s+h}$ would follow $X_{t+h}$ similar to the way $X_{s}$ follows $X_t$.

Now due the way stationarity is defined this holds for any number of points. So in some sense function should look similar at different locations. Note that this depends on the distributional properties of the process. White noise is a stationary process too and it does not look similar at different points in the usual common sense.

• Ok, I think I understood now. So, looking at the negative squared exponential now, if we look at it defining the covariance between two random variables x and y, it is a function of x-y rather than x and y and hence it is invariant to the shift. – Luca Oct 15 '14 at 14:12
• @Luca No, the covariance between two random variables $X = X(t_1)$ and $Y =X(t_2)$ (samples at different times $t_1$ and $t_2$) is a function of $t_1-t_2$, not of $X-Y$, and this covariance is the same for random variables $W=X(t_1+\tau)$ and $Z = X(t_2+\tau)$. The invariance to shift is a shift (by $\tau$ here) in the times of the two samples, not to the difference $W-Z$ of the two samples. – Dilip Sarwate Oct 15 '14 at 14:50
• Ahhhhh yes, of course. So it is a function of the distance in time/space/whatever between the variables rather than the values they take at those instances. Subtle! Many thanks. – Luca Oct 15 '14 at 15:16