A "systematic" part of a random time series component?

Question

I've seen at least 3 sources on time series* state that the component of a series that is variously called random, stochastic, or noise (something clearly separate from any deterministic, patterned component) itself consists of 2 parts, a systematic part and an unsystematic part. I can't for the life of me figure out how the random part can in turn have a systematic part. I can see how the series can have unsystematic and systematic parts, but not how the random component of the series could have both of these.

*For example, Burns and Grove, The Practice of Nursing Research (2005, p. 475-6).

Answer 1

The Burns reference that you are quoting seems to dividing the stochastic part into autocorrelation error, which is a byproduct of any time series analysis (and is systematic), vs. truly random error which is uncontrollable.

-Ralph Winters

Answer 2

"random" is often used as if it was a real property of the data under study, where it should be replaced with "uncertain". To give an example, if I ask you what how much money you earned over the past month, and you don't tell me, it is not "random", but just uncertain. However, treating the uncertainty as if it was random allows you to make some useful conclusions.

The noise is not "random" per se, but given that we usually have limited knowledge of how each particular piece of "noise" is generated, assuming that it is random can be useful.

Now whenever you fit a model to some data, you will have residuals from that model. And if treating the "noise" as if it was random was a good idea, then the residuals from the model should be consistent with whatever definition of "randomness" you have used in fitting the model. If they are not, then basically the time series is telling you that the "randomness" you assumed is not a good description of what it actually happening, and it gives you a clue as to what a better description might be.

For example, if I fit a linear relationship for the systematic part, but it is actually quadratic, then the so-called "random" noise will not look random at all, rather it will contain the squared component of the systemtatic part.

To make it even more concrete, suppose that your response $Y$ is a deterministic function of $X$, say $Y=3+2X+X^2$. Now, because you don't know this function you suppose that $Y=\alpha+\beta X + error$, and because you have no reason to doubt the model prior to seeing the data, you assume that the error is just "random noise" (usually $N(0,\sigma^2)$). However, once you actually fit your data and look at the residuals, they will all line up as an exact quadratic function of the residuals. Thus there is a systematic component to the "noise" (in fact the "noise" is entirely systematic). This is basically "Nature" telling "You" that you model is wrong, and gives a clue as to how it could be improved.

The same kind of thing is happening in the time series. you could just replace the model above with $Y_{1}=1,Y_{2}=6,Y_{3}=0.5,Y_{4}=10,Y_{5}=3,Y_{6}=10$ and for $t\geq7$ have $Y_t=10+2 Y_{t-1} -5Y_{t-1}^3 + 2Y_{t-5}$ and the same kind of thing would happen.

Answer 3

Systematic and unsystematic are rather ambiguous terms. One of the possible explanations is given by @probabilityislogic. Another may be given here. Since the context you gave is time series, I think this might be related to Wold's theorem. Unfortunately wikipedia text captures the essence, but does not go into the details of which part is systematic and non systematic.

I did not manage to find appropriate link to refer to, so I will try give some explanation based on the book I have. This subject is also discussed in this book. I will not give precise and rigorous definitions, since they involve Hilbert spaces and other graduate mathematics stuff, which I think is not really necessary to get the point across.

Each covariance-stationary process $\{X_t,t\in \mathbb{Z}\}$ can be uniquely decomposed into two stationary proceses: $X_t=M_t+N_t$, singular $M_t$ and regular $N_t$.

Singular and regular processes are defined via their prediction properties. In stationary process theory the prediction of process $X_t$ at time $t$ is formed from linear span of its history $(X_s,s<t)$. Singular processes are processes for which the prediction error:

$$E(\hat{X}_t-X_t)^2$$

is zero. Such processes sometimes are called deterministic, and in your context can be also called systematic. The most simple example of such process is $X_t=\eta$ for all $t$ and $\eta$ some random variable. Then the linear prediction of $X_t$ based on its history will always be $\eta$. The error of such prediction as defined above would be zero.

Regular stationary processes on the other hand cannot be predicted without error from their history. It can be shown that the stationary process $N_t$ is regular if and only if it admits $MA(\infty)$ decomposition. This means that there exists white-noise sequence $(\varepsilon_t)$ such that

$$N_t=\sum_{t=0}^{\infty}c_n\varepsilon_{t-n}.$$

where coefficients $c_n$ are such, that the equality holds. These processes sometimes are called non-deterministic, or probably non-systematic in your case.