Time series model is defined as :

A time series model specifies the joint distribution of the sequence ${\{X_t}\}$ of random variables. For example:$$P[X_1\le x_1,\ldots,X_t\le x_t]$$ for all $t$ and $x_1,\ldots,x_t$.

I have not understood the definition.

  • First , why have they mentioned joint distribution ?

  • Second , What is ${\{X_t}\}$ ? Is that, say, at $t$ time point the sale of a shop ?

  • Third, Why is this the form of a cdf , i.e., $$P[X_1\le x_1,\ldots,X_t\le x_t]$$ for all $t$ and $x_1,\ldots,x_t$

instead of $$P[X_1= x_1,\ldots,X_t= x_t]$$ for all $t$ and $x_1,\ldots,x_t$.

  • $\begingroup$ Could you provide a reference for your quote for clarity? $\endgroup$
    – Tim
    Mar 20 '15 at 13:56
  • $\begingroup$ @Tim I have mentioned the link in my post. Please see the edit. $\endgroup$
    – time
    Mar 20 '15 at 14:00

They mention joint distribution because with analyzing time-series you are interested in the whole series and its changes in time, so in its joint distribution. Notice that on the slides you refer to it is said that $X_1, X_2,...$ is a stochastic process, that means it is a

collection of random variables, representing the evolution of some system of random values over time. This is the probabilistic counterpart to a deterministic process (or deterministic system). Instead of describing a process which can only evolve in one way (as in the case, for example, of solutions of an ordinary differential equation), in a stochastic or random process there is some indeterminacy: even if the initial condition (or starting point) is known, there are several (often infinitely many) directions in which the process may evolve. (Wikipedia)

so it is not just a collection of random variables, but a system that evolves. Because of that we need a model that describes a whole system, so the joint distribution of all the states.

$X_t$ is a random variable $X$ at point in time $t$, so your example is correct - it could be a time point from a dataset about shop sales, while $x_t$ is the actual occurrence of $X$ at time $t$.

$P[X_1 \leq x_1, ..., X_t \leq x_t]$ is mentioned rather then $P[X_1 = x_1, ..., X_t = x_t]$ since it is a function that describes both continuous and discrete variables while probability mass function ($P[X_1 = x_1, ..., X_t = x_t]$) describes only discrete variables. It is a case because for continuous variables $P[X = x] = 0$ since there is infinite number of possible $x$'s and their probabilities get infinitely small.

  • 1
    $\begingroup$ Tiny hitch: probabilities of concrete $x$'s are infinitely small, not the $x$'s themselves. Also, CDF works for both continuous as well as discrete variables while PMF works only for discrete variables, so perhaps that's why CDF is used (we do not really need to omit the case of discrete variables, do we?). $\endgroup$ Mar 20 '15 at 14:48
  • $\begingroup$ @RichardHardy right! Thanks for pointing that! Corrected. $\endgroup$
    – Tim
    Mar 20 '15 at 18:35
  • $\begingroup$ From Wikipedia definition, could you please tell me what is meant by evolution of "some system" of random values? $\endgroup$
    – time
    Mar 21 '15 at 1:02
  • $\begingroup$ I know that for continuous variable $P[X=x]=0$. But i can't visualize if $P[X=x]=0$, there is infinite number of possible $x$'s. And also why do their probabilities get infinitely small ? $\endgroup$
    – time
    Mar 21 '15 at 1:10
  • $\begingroup$ @tree ok, but I think this is a totally different topic on basics of probability theory. I think it would be better to ask about it in new question. Also posting it as another question will make the answer more accessible for others who will search for it in the future - they won't search for "time-series" but rather for "probability", so better not to hide this question in time-series question. $\endgroup$
    – Tim
    Mar 21 '15 at 7:26

Perhaps a more intuitive way to look at the definition is to start with a single random variable $X$ taking values on $\bf R$. Without going into technicalities, when can you say you have fully specified the behavior of the random variable? When you know its CDF. Given the CDF, you can obtain every possible statistic: mean, standard deviation, quantiles, etc. If the CDF is a "smooth" function (differentiable over, say, an interval), you can also obtain the probability density (by taking the derivative of the CDF). Note: if the CDF is smooth, the probability that X takes value x, $P(X=x) = 0$, is zero! That's the rationale for using a CDF: because it works for discrete and continuous (and weirder) values of $X$.

Now, take a step further and think of a pair of random variables $(X, Y)$, describing the occurrence of a point on the plane. When can you say you know their behaviour? When you know the joint CDF $F(x,y)$, i.e., Probability that $X$ takes values smaller or equal than a number $x$, and $Y$ takes values smaller or equal than $y$. From the joint CDF $F$ you can always get the single (marginal) CDF for $X$ and $Y$. Intuitively, $F_X(X)= F(x, \infty)$, i.e., $P(X\le x)$ and $Y$ taking any value less than infinity. But, the individual CDF of $X$ and $Y$ are not enough to characterize the probability distribution of $(X,Y)$. You can find an example yourself when $X$ and $Y$ are Bernoulli (e.g., they take values Head and Tail). Say you toss one coin and give to $X$ and $Y$ the same value, i.e., the outcome of the toss. The individual CDFs for $X$ and $Y$ are the same $P(X=Head)=P(X=Tail)=1/2$; $P(Y=Head)=P(Y=Tail)=1/2$. The probability that they have the same value is 1, because we defined them to be identical. Now, choose a different definition: you toss a coin for $X$ and a coin for $Y$. The individual distributions are the same as before, but now $P(X=Y)=1/2$! So you really need the joint distribution.

The last step is to consider a vector of random variables of arbitrary length: $(X_1, X_2, X_3, \ldots)$. A time series is just a random vector. The index is interpreted as time, so, $X_t$ could be store sales on hour $t$, or the number of clients in line at a store, or the price of a stock at minute $t$.

But to describe the behaviour of this vector, you still need the joint CDF (your first question) and not the probability that it takes a specific vector. And the interpretation of the $t$-the element of the vector is usually "what happens of some uncertain event at time $t$.

  • $\begingroup$ Could you please tell what is meant by identical ? $\endgroup$
    – time
    Mar 21 '15 at 0:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.