Monte Carlo Integration Results in Heavy Tailed Distribution

Question

I am running a Monte Carlo simulation that results in an heavy-tailed distribution. The image below shows the distribution of 1,200 runs of the Monte Carlo simulation, where each run consists of integrating over $M$ = 12,000 randomly drawn paths of $ \mathcal{X}_m = \left\{X_{n,m},s_{n,m}\right\}_{n=1}^N$.

The quantity I am simulation is an expectation of a definite sum of exponentials, where I know the sum converges as $N \rightarrow \infty$.

$$\mathbb{E} \left[ S\left(\mathcal{X}\right) \middle| X_1, s_1 \right] = \mathbb{E} \left[\exp\left( X_1\right) + \exp\left(X_1 + X_2\right) + \cdots + \exp\left(X_1 + X_2 + \cdots + X_N\right) \middle| X_1, s_1 \right]. $$

$X_n$ is a Markov-switching Autoregressive process with Gaussian errors:

$$X_n = \alpha_{s_n} + \rho_{s_n} X_{n-1} + \sigma_{s_n} \epsilon_n,$$

where $\epsilon_n \sim N(0,1)$, $s_n \in \{1,2\} \sim \Pi$, and $\Pi$ is a transition matrix.

For a single realization of what is presented in the histogram, I compute $S(\mathcal{X}_m)$ for each of the $M$ simulated paths. And then I simply take an average over the $M$ realizations,

$$\mathbb{E}\left[S\left(\mathcal{X}\right) \middle| X_1, s_1 \right] \approx \frac{\sum_{m=1}^M S(\mathcal{X}_m)}{M}.$$

I believe Monte Carlo should result asymptotically in a normal distribution, but this resembles a log-normal distribution. How would I diagnose this issue? How should I change my simulation strategy?

I've proved the sum converges. The proof boils down to the unconditional mean of $X_n <0$, and sum inside the exponential goes to negative infinity faster than one-half the unconditional variance.

The instability occurs when the sum inside the exponential terms is greater than 0 for a few periods (the process is persistent). Even though it will eventually converge to -$\infty$, and can blow up temporarily.

Answer 1

I am still not sure I understand what you are trying to simulate, but below I have tried to answer the question to how I think it works.

Simulation setup

The simulations depends on three quantities:

• $K$, the number of simulations of the $(X_1, s_1)$. That is $(x_1,\tilde s_1)\sim P_{X_1, s_1}$
• $M$, the number of simulations of $S_N(X)|(X_1,s_1)=(x_1,\tilde s_1)$ per simulated value $(x_1,\tilde s_1)$.
• $N$, the length of the chain $X_1, \dots, X_N$.

And results in $K$ realizations of the conditional expectation (as a random variable), $E[S_N(X)|X_1,s_1]$. Let $\hat P_{K,M,N}$ denote the actual distribution of the simulations and let $P$ be the hypothesized asymptotic distribution. Let $d$ be a favorite distance measure between distributions. The goal of simulations is to determine if $d(\hat P_{K,M,N},P)$ can be made infinitesimally small by increasing $K,M$ and $N$. That can be a daunting task, so let's break it down.

Size of $K$

For a given size of $K$, it is known how similar the simulated distribution should be to the desired asymptotic distribution. One can simply simulate from the desired asymptotic distribution $K$ times, obtaining a empirical distribution $\hat P_K$ and compute $d(\hat P_K, P)$.

Size of $M$

The size of $M$ should be so big that the variance of $S_N(X)|(X_1,s_1)$ becomes negligible. I don't know if it actually has a finite variance, but if one could get some bound on it, $L$, it would be known that the variance of $\hat E_M[S_N(X)|(X_1,s_1)]$ is less than $L/M$. Then, for a given $K$ and $M$, we can simulate $K$ realizations of the desired asymptotic distribution and add $N(0,L/M)$ noise, obtaining an empirical distribution $\hat P_{K,M}$ and then compute $d(\hat P_{K,M}, P)$.

Size of $N$

For a given $K$ and $M$, I suggest that simulations are used for increasing $N$ to show that $d(\hat P_{K,M,N}, P)$ can be made as small as $d(\hat P_{K,M},P)$. The simulations for different values of $N$ doesn't need to be independent to make a convincing argument, so one can simply continue the same $K\cdot M$ chains.

In practice one may want to simulate a distribution for $d(\hat P_{K,M}, P)$ instead of just a single number. $K$ and $M$ should be chosen such that $d(\hat P_{K,M},P)$ becomes so small that it is convincing when it is shown that $d(\hat P_{K,M,N}, P)$ is of similar size.