5
$\begingroup$

Suppose that for all $t\in\mathbb{Z}$, the distribution of $x_t|x_{t-1},x_{t-2},\dots$ has probability density function $f(x_t|x_{t-1})$, where $x_t,x_{t-1}\in\mathbb{R}^n$.

Suppose further that the unconditional (stationary) distribution of $x_t$ is that of a standard $n$-dimensional Gaussian distribution (i.e. mean zero, covariance identity).

Can anything be inferred about the tails of $f(\cdot|x_{t-1})$? Intuitively, it seems like a stationary distribution ought to have at least as fat tails as the conditional distribution. Is this a theorem?

$\endgroup$
7
  • $\begingroup$ The tail of $f(\,.\,|x_{t-1})$ depends on $x_{t-1}$, making it difficult to compare it to the tail of $x_t$. However the average tail can be assessed by taking the expectation of the conditional survival $S(x|x_{t-1})$ for a fixed large value of $x$, which is easily found to be the stationary survival $S(x)$. So in a sense the average tail is the tail of the stationary distribution. $\endgroup$
    – Yves
    Dec 17, 2017 at 18:54
  • $\begingroup$ Can anything be said about tail indices? (Which are not moments.) $\endgroup$
    – cfp
    Dec 18, 2017 at 18:04
  • $\begingroup$ Well in your question, the process is only assumed to be marginally Gaussian. If it is a Gaussian Process (GP), then for any finite margin tail independence holds. I have no example where a Gaussian stationary distribution arises from a non-GP Markov process. At least for $n=1$ your question makes sense for any distribution. I guess that the results used in the conditional approach of Heffernan and Tawn could help. $\endgroup$
    – Yves
    Dec 18, 2017 at 19:53
  • 1
    $\begingroup$ In the univariate case, it's trivial to generate examples. Take any stationary Markov process $y_t$. Let $F$ be the CDF of its stationary distribution. Then define $x_t:=\Phi^{-1}(F(y_t))$. Can you prove the claim in this case at least? $\endgroup$
    – cfp
    Dec 20, 2017 at 12:40
  • 1
    $\begingroup$ Since the intuition is yours you should write the new question, possibly linking to this one and to a definition of the tail index. I am also curious about the answer and the proof. $\endgroup$
    – Yves
    Dec 23, 2017 at 15:29

2 Answers 2

2
$\begingroup$

I will elaborate on a more general -yet vague- following statement: a mixture of continuous distributions has a tail which is heavier than that of its components. For that aim, consider a mixture of absolutely continuous distributions $f(y \vert \theta)$ with "weight" distribution $\pi(\theta)$
\begin{equation} \tag{1} f(y) = \int_{\Theta} f(y \vert \theta)\, \pi(\theta) \, \text{d} \theta, \end{equation} and assume here that the support $\Theta$ of $\pi$ is a real interval. With slight changes in the notations, this context applies to the stationary distribution of a Markov chain, the densities $f$ and $\pi$ then being identical.

I will focus on the case where the distribution -say of a r.v. $Y$- has $\infty$ as its upper end-point and is Regularly Varying (RV) with index $\alpha \geq 0$ which means that $$ S(y) \sim y^{-\alpha} \mathcal{L}(y), \quad y \to \infty $$ where $S(y) := \text{Pr}\{Y > y\}$ is the survival function, and $\mathcal{L}(y)$ is slowly varying function, that is: $\lim_{y \to \infty} \mathcal{L}(ty) / \mathcal{L}(y) = 1$ holds for every finite $t >0$. I use here the definition of the report by T. Mikosch cited below, although the tail index is usually defined as $\xi := 1/\alpha$. So, the smaller $\alpha$, the thicker the tail. Without loss of generality, we can assume that $Y \geq 0$. An interesting characterisation of $\alpha$ is in terms of moments: $$ \begin{cases} \mathbb{E}[Y^{\beta}] < \infty & \text{if } \beta < \alpha, \\ \mathbb{E}[Y^{\beta}] = \infty & \text{if } \beta > \alpha, \end{cases} $$ see Prop 1.3.2 in Mikosch. Using Tonelli's theorem $$ \text{E}[Y^\beta]= \int_0^\infty y^{\beta} \left[ \int_{\Theta} f(y \vert \theta) \,\pi(\theta) \, \text{d} \theta \right] \text{d}y = \int_{\Theta} \left\{\int_0^\infty y^{\beta} f(y \vert \theta)\, \text{d} y \right\} \pi(\theta) \, \text{d} \theta. $$ Now assume that $f(y \vert \theta)$ is RV with index $\alpha(\theta)$.

  • Assume that $\alpha(\theta)$ is constant w.r.t. $\theta$. If $\beta > \alpha(\theta)$, then the integral between the curly brackets {} is infinite for all $\theta$, and the left hand side is also infinite. So $\beta > \alpha(\theta)$ implies that $\beta \geq \alpha$, which tells that $\alpha \leq \alpha(\theta)$.

  • Assume that $\alpha(\theta)$ varies smoothly with $\theta$. If $\beta > \min_\theta \alpha(\theta)$ then there exist a real interval $I \subset \Theta$ with positive width such that $\beta > \alpha(\theta)$ for every $\theta \in I$, implying that the integral between the curly brackets is infinite, hence that the left hand side is infinite. As before we conclude that $\alpha \leq \min_{\theta} \alpha(\theta)$.

So we see that the mixture has a tail which is at least as heavy as that of the heaviest-tailed component $f(y\vert \theta)$. This could be generalised to $\alpha = \infty$ with a few changes in the definition for RV then. Moreover, by replacing the moments $\text{E}[Y^\beta]$ by exponential moments $\text{E}[e^{\beta Y }]$, thin-tailed distributions can be compared similarly.

Beside the stationary distribution of a Markov chain in the question, there are many other examples ot this "tail-broadening" phenomenon.

  • The heavy-tailed Lomax distribution (with $\alpha > 0$) can be obtained by taking $\pi(\theta)$ to be gamma with shape $\alpha$ and $f(y \vert \theta)$ to be exponential with rate $\theta$, for which $\alpha = \infty$. So, the Lomax distribution is a mixture of thin-tailed distributions.

  • In the Bayes context, we can take $\pi(\theta)$ to be $\pi(\theta \vert \mathbf{y}_{\text{obs}})$ for some observed vector $\mathbf{y}_{\text{obs}}$. We then get that the tail of the predictive posterior distribution $f(y \vert \mathbf{y}_{\text{obs}})$ is heavier than that of the likelihood $f(y \vert \theta)$, due to the uncertainty on $\theta$ conditional on $\mathbf{y}_{\text{obs}}$. Most predictive distributions are heavy-tailed as are Student and Fisher-Snedecor distributions.

Mikosch, T (1999) Regular Variation, Subexponentiality and Their Applications in Probability Theory, Eindhoven University of Technology.

$\endgroup$
1
  • 1
    $\begingroup$ Great answer! I never did get round to writing a new question, but it's wonderful you answered here anyway. $\endgroup$
    – cfp
    Apr 11, 2018 at 20:46
1
$\begingroup$

Can anything be inferred about the tails of $f(\cdot|x_{t-1})$?

By stationarity $$ \int f(x_t \mid x_{t-1}) \pi(x_{t-1}) dx_{t-1} =\pi(x_{t}). $$ By your assumptions of normality: $$ \int f(x_t \mid x_{t-1}) \phi^n(x_{t-1}) dx_{t-1} =\phi^n(x_{t}), \tag{2} $$ where $\phi^n(x_t) = \prod_{i=1}^n \phi(x_{ti})$. This (probably) means that $$ f(x_t \mid x_{t-1}) = N(x_t ; Ax_{t-1}, \Sigma), $$ for any $A$, $\Sigma$ such that $\Sigma + AA' = I$. So both densities have a standard kurtosis. Actually, I am not 100% certain about how multivariate kurtosis is defined.

Edit:

This won't completely satisfy what you're looking for but if you look at the LHS of (2) you get $$ (2 \pi)^{-1} \det(\Sigma)^{-1/2} \int \exp\left[-\frac{1}{2}\left\{(x_t - A x_{t-1})'\Sigma^{-1}(x_t - Ax_{t-1})- x_{t-1}'x_{t-1} \right\} \right] dx_{t-1} \tag{3} $$ inside the curly braces turns out to be $$ x_t'\Sigma^{-1}x_t - 2x_{t-1}'A'\Sigma^{-1}x_t + x_{t-1}'\left[ A'\Sigma^{-1}A-I\right] x_{t-1} $$ and you can complete the square and re-arrange this as $$ (x_{t-1} - Bx_t)'C^{-1}(x_{t-1} - Bx_t) - x_tB'Bx_t + x_t'\Sigma^{-1}x_t $$ where $C^{-1} = A'\Sigma^{-1}A-I$ and $B = \left(A'\Sigma^{-1}A-I\right)^{-1}A'\Sigma^{-1} $. Completing the square like this allows us to integrate (3) by recognizing a Gaussian density. You integrate out $x_{t-1}$ and what should be left is a Gaussian density for $x_t$ with mean $0$ and variance $I$.

This is technically only a sufficient condition for the marginals to be Gaussian, and not the other way around, which is what you're looking for, but you can start playing around with what happens if you don't have this. The integral would be pretty tough to get.

In the univariate case, the tails you're after, $E[X_t^4 \mid x_{t-1}]$, on average have to be $3$ by the law of total expectation. But with this example, we just assume these are all $3$, regardless of $x_{t-1}$. If you find an example that's different from this one, where $x_{t-1}$ influences more than just the mean of $x_t \mid x_{t-1}$, then I would be interested in hearing about it.

$\endgroup$
4
  • $\begingroup$ Thanks. Could you clarify why your second equation implies your third? Why should $f$ be Gaussian? This convolutional property of Gaussian distributions is not one I recognise. How is it proven? $\endgroup$
    – cfp
    Dec 16, 2017 at 9:37
  • $\begingroup$ It is certainly not the case that normal marginal distributions implies joint normality. For example, take a multivariate t-distribution with non-diagonal covariance, then apply the CDFs of the marginal distributions (elementwise), then apply an inverse normal-CDF (elementwise). The result has normal marginal, but is not normal. $\endgroup$
    – cfp
    Dec 16, 2017 at 14:24
  • $\begingroup$ @cfp see my edits $\endgroup$
    – Taylor
    Dec 16, 2017 at 16:10
  • $\begingroup$ If $f$ were Gaussian, then yes your restriction on the mean and covariance of $x_t|x_{t-1}$ would be correct (and is immediately clear). I wasn't asking for (3) etc., I was asking for a proof that $f$ had to be Gaussian. I see now that you were merely assuming Gaussianity (so there was no proof). Your idea of using the law of iterated expectations for moments is a start though. $\endgroup$
    – cfp
    Dec 16, 2017 at 16:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.