Skip to main content
added 119 characters in body
Source Link
Richard Hardy
  • 69.5k
  • 13
  • 126
  • 278

You have stumbled upon a problem that plagues most of time series analyses. A time series $\{X_t\}$ is an ordered collection of random variables: $X_1,X_2,\dots$. We observe a single observation for a chunk of them: $(x_1,x_2,\dots,x_T)$. Of course, it is impossible to conclude much about the distribution of the random variable $X_t$ from only a single observation $x_t$.

Therefore, we assume some relationship between these distributions, e.g. that all the distributions are the same and independent of each other (the i.i.d. model) or that they are almost the same but dependent with a certain simple structure of how their means (ARMA) or variances (GARCH) are shifting over time (ARMAmore details here). ThisThis makes an observation $x_s$ informative of the distribution of the random variable $X_t$.

In the i.i.d. case, we can treat $(x_1,x_2,\dots,x_T)$ as a sample from $X_1$ and we can obtain an estimate of the mean, the variance and any other features of $X_1$ from the sample. Since $X_2$ and all other random variables have the same distribution, learning about $X_1$ is enough. In the GARCH case, the functional form of the distribution is the same for all $X$s, but the variance is changing as we move from $X_1$ to $X_2$ to ... according to a simple rule. This is enough to be able to learn the parameters of the simple rule, fit the variances of $X_1$ to $X_T$ and predict the variance of $X_{T+1}$. How this is done on the implementation level is not trivial, but the idea behind is based on the logic above.

You have stumbled upon a problem that plagues most of time series analyses. A time series $\{X_t\}$ is an ordered collection of random variables: $X_1,X_2,\dots$. We observe a single observation for a chunk of them: $(x_1,x_2,\dots,x_T)$. Of course, it is impossible to conclude much about the distribution of the random variable $X_t$ from only a single observation $x_t$.

Therefore, we assume some relationship between these distributions, e.g. that all the distributions are the same and independent of each other (the i.i.d. model) or that they are almost the same but dependent with a certain simple structure of how their means (ARMA) or variances (GARCH) are shifting over time (ARMA). This makes an observation $x_s$ informative of the distribution of the random variable $X_t$.

In the i.i.d. case, we can treat $(x_1,x_2,\dots,x_T)$ as a sample from $X_1$ and we can obtain an estimate of the mean, the variance and any other features of $X_1$ from the sample. Since $X_2$ and all other random variables have the same distribution, learning about $X_1$ is enough. In the GARCH case, the functional form of the distribution is the same for all $X$s, but the variance is changing as we move from $X_1$ to $X_2$ to ... according to a simple rule. This is enough to be able to learn the parameters of the simple rule, fit the variances of $X_1$ to $X_T$ and predict the variance of $X_{T+1}$. How this is done on the implementation level is not trivial, but the idea behind is based on the logic above.

You have stumbled upon a problem that plagues most of time series analyses. A time series $\{X_t\}$ is an ordered collection of random variables: $X_1,X_2,\dots$. We observe a single observation for a chunk of them: $(x_1,x_2,\dots,x_T)$. Of course, it is impossible to conclude much about the distribution of the random variable $X_t$ from only a single observation $x_t$.

Therefore, we assume some relationship between these distributions, e.g. that all the distributions are the same and independent of each other (the i.i.d. model) or that they are almost the same but dependent with a certain simple structure of how their means (ARMA) or variances (GARCH) are shifting over time (more details here). This makes an observation $x_s$ informative of the distribution of the random variable $X_t$.

In the i.i.d. case, we can treat $(x_1,x_2,\dots,x_T)$ as a sample from $X_1$ and we can obtain an estimate of the mean, the variance and any other features of $X_1$ from the sample. Since $X_2$ and all other random variables have the same distribution, learning about $X_1$ is enough. In the GARCH case, the functional form of the distribution is the same for all $X$s, but the variance is changing as we move from $X_1$ to $X_2$ to ... according to a simple rule. This is enough to be able to learn the parameters of the simple rule, fit the variances of $X_1$ to $X_T$ and predict the variance of $X_{T+1}$. How this is done on the implementation level is not trivial, but the idea behind is based on the logic above.

Source Link
Richard Hardy
  • 69.5k
  • 13
  • 126
  • 278

You have stumbled upon a problem that plagues most of time series analyses. A time series $\{X_t\}$ is an ordered collection of random variables: $X_1,X_2,\dots$. We observe a single observation for a chunk of them: $(x_1,x_2,\dots,x_T)$. Of course, it is impossible to conclude much about the distribution of the random variable $X_t$ from only a single observation $x_t$.

Therefore, we assume some relationship between these distributions, e.g. that all the distributions are the same and independent of each other (the i.i.d. model) or that they are almost the same but dependent with a certain simple structure of how their means (ARMA) or variances (GARCH) are shifting over time (ARMA). This makes an observation $x_s$ informative of the distribution of the random variable $X_t$.

In the i.i.d. case, we can treat $(x_1,x_2,\dots,x_T)$ as a sample from $X_1$ and we can obtain an estimate of the mean, the variance and any other features of $X_1$ from the sample. Since $X_2$ and all other random variables have the same distribution, learning about $X_1$ is enough. In the GARCH case, the functional form of the distribution is the same for all $X$s, but the variance is changing as we move from $X_1$ to $X_2$ to ... according to a simple rule. This is enough to be able to learn the parameters of the simple rule, fit the variances of $X_1$ to $X_T$ and predict the variance of $X_{T+1}$. How this is done on the implementation level is not trivial, but the idea behind is based on the logic above.