5
$\begingroup$

I've seen at least 3 sources on time series* state that the component of a series that is variously called random, stochastic, or noise (something clearly separate from any deterministic, patterned component) itself consists of 2 parts, a systematic part and an unsystematic part. I can't for the life of me figure out how the random part can in turn have a systematic part. I can see how the series can have unsystematic and systematic parts, but not how the random component of the series could have both of these.

*For example, Burns and Grove, The Practice of Nursing Research (2005, p. 475-6).

Improve this question
$\endgroup$

3 Answers 3

Reset to default
6
$\begingroup$

The Burns reference that you are quoting seems to dividing the stochastic part into autocorrelation error, which is a byproduct of any time series analysis (and is systematic), vs. truly random error which is uncontrollable.

-Ralph Winters

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ @Ralph, +1 for looking up the reference. I should note however that if this is truly how the authors are dividing stochastic part it is a bit misleading. Suppose you have autoregressive model $Y_t=\rho Y_{t-1}+\varepsilon_t$. The division is that $\rho Y_{t-1}$ is systematic and $\varepsilon_t$ is non-systematic part. But if we substitute $Y_{t-1}=\rho Y_{t-2}+\varepsilon_{t-1}$ into the first equation and repeat the process indefinitely we will get that $Y_t=\sum_{s=0}^{\infty} \rho^{s}\varepsilon_{t-s}$, so there is actually no systematic part according to the original definition. $\endgroup$
    – mpiktas
    Jan 31, 2011 at 8:28
3
$\begingroup$

"random" is often used as if it was a real property of the data under study, where it should be replaced with "uncertain". To give an example, if I ask you what how much money you earned over the past month, and you don't tell me, it is not "random", but just uncertain. However, treating the uncertainty as if it was random allows you to make some useful conclusions.

The noise is not "random" per se, but given that we usually have limited knowledge of how each particular piece of "noise" is generated, assuming that it is random can be useful.

Now whenever you fit a model to some data, you will have residuals from that model. And if treating the "noise" as if it was random was a good idea, then the residuals from the model should be consistent with whatever definition of "randomness" you have used in fitting the model. If they are not, then basically the time series is telling you that the "randomness" you assumed is not a good description of what it actually happening, and it gives you a clue as to what a better description might be.

For example, if I fit a linear relationship for the systematic part, but it is actually quadratic, then the so-called "random" noise will not look random at all, rather it will contain the squared component of the systemtatic part.

To make it even more concrete, suppose that your response $Y$ is a deterministic function of $X$, say $Y=3+2X+X^2$. Now, because you don't know this function you suppose that $Y=\alpha+\beta X + error$, and because you have no reason to doubt the model prior to seeing the data, you assume that the error is just "random noise" (usually $N(0,\sigma^2)$). However, once you actually fit your data and look at the residuals, they will all line up as an exact quadratic function of the residuals. Thus there is a systematic component to the "noise" (in fact the "noise" is entirely systematic). This is basically "Nature" telling "You" that you model is wrong, and gives a clue as to how it could be improved.

The same kind of thing is happening in the time series. you could just replace the model above with $Y_{1}=1,Y_{2}=6,Y_{3}=0.5,Y_{4}=10,Y_{5}=3,Y_{6}=10$ and for $t\geq7$ have $Y_t=10+2 Y_{t-1} -5Y_{t-1}^3 + 2Y_{t-5}$ and the same kind of thing would happen.

Improve this answer
$\endgroup$
2
  • $\begingroup$ +1 with the request for clarification: the "systematic" part of the residual should be nigh undetectible, otherwise it is an indication that the model is not satisfactory: a systematic effect -- whether real or an artifact of analysis -- has leaked out of the model, as it were, and the model needs to be modified to avoid this. $\endgroup$
    – Wayne
    Sep 26, 2011 at 16:33
  • $\begingroup$ @wayne, this may or may not be the case. If you have enough resources/time, you could probably find some justification for any part of the noise being "systematic". This is basically an "over-fitting" problem, and also one of bias-variance trade off. Taking account of a supposed systematic effect may lead to a better fit to the current data set, but could well lead to a worse fit in other data sets. $\endgroup$
    – probabilityislogic
    Sep 26, 2011 at 21:09
3
$\begingroup$

Systematic and unsystematic are rather ambiguous terms. One of the possible explanations is given by @probabilityislogic. Another may be given here. Since the context you gave is time series, I think this might be related to Wold's theorem. Unfortunately wikipedia text captures the essence, but does not go into the details of which part is systematic and non systematic.

I did not manage to find appropriate link to refer to, so I will try give some explanation based on the book I have. This subject is also discussed in this book. I will not give precise and rigorous definitions, since they involve Hilbert spaces and other graduate mathematics stuff, which I think is not really necessary to get the point across.

Each covariance-stationary process $\{X_t,t\in \mathbb{Z}\}$ can be uniquely decomposed into two stationary proceses: $X_t=M_t+N_t$, singular $M_t$ and regular $N_t$.

Singular and regular processes are defined via their prediction properties. In stationary process theory the prediction of process $X_t$ at time $t$ is formed from linear span of its history $(X_s,s<t)$. Singular processes are processes for which the prediction error:

$$E(\hat{X}_t-X_t)^2$$

is zero. Such processes sometimes are called deterministic, and in your context can be also called systematic. The most simple example of such process is $X_t=\eta$ for all $t$ and $\eta$ some random variable. Then the linear prediction of $X_t$ based on its history will always be $\eta$. The error of such prediction as defined above would be zero.

Regular stationary processes on the other hand cannot be predicted without error from their history. It can be shown that the stationary process $N_t$ is regular if and only if it admits $MA(\infty)$ decomposition. This means that there exists white-noise sequence $(\varepsilon_t)$ such that

$$N_t=\sum_{t=0}^{\infty}c_n\varepsilon_{t-n}.$$

where coefficients $c_n$ are such, that the equality holds. These processes sometimes are called non-deterministic, or probably non-systematic in your case.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Don't the $c_t$ need to form an absolutely convergent sum? i.e. $\sum_{t=0}^{\infty}|c_t| < \infty$ as well $\endgroup$
    – probabilityislogic
    Jan 30, 2011 at 10:03
  • $\begingroup$ @probabilityislogic, I intentionaly wrote non-rigorous mathematical text, since strict definition of singular and regular processes involves Hilbert spaces, which I do not think are really necessary to get the point accros. I will update the answer to reflect that. Concerning $c_t$, it does not need to be summable. The equation must hold in $L^2$ sense, hence summability of $c_t^2$ is sufficient. $\endgroup$
    – mpiktas
    Jan 30, 2011 at 11:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.