# Monte Carlo Integration Results in Heavy Tailed Distribution

I am running a Monte Carlo simulation that results in an heavy-tailed distribution. The image below shows the distribution of 1,200 runs of the Monte Carlo simulation, where each run consists of integrating over $$M$$ = 12,000 randomly drawn paths of $$\mathcal{X}_m = \left\{X_{n,m},s_{n,m}\right\}_{n=1}^N$$.

The quantity I am simulation is an expectation of a definite sum of exponentials, where I know the sum converges as $$N \rightarrow \infty$$.

$$\mathbb{E} \left[ S\left(\mathcal{X}\right) \middle| X_1, s_1 \right] = \mathbb{E} \left[\exp\left( X_1\right) + \exp\left(X_1 + X_2\right) + \cdots + \exp\left(X_1 + X_2 + \cdots + X_N\right) \middle| X_1, s_1 \right].$$

$$X_n$$ is a Markov-switching Autoregressive process with Gaussian errors:

$$X_n = \alpha_{s_n} + \rho_{s_n} X_{n-1} + \sigma_{s_n} \epsilon_n,$$

where $$\epsilon_n \sim N(0,1)$$, $$s_n \in \{1,2\} \sim \Pi$$, and $$\Pi$$ is a transition matrix.

For a single realization of what is presented in the histogram, I compute $$S(\mathcal{X}_m)$$ for each of the $$M$$ simulated paths. And then I simply take an average over the $$M$$ realizations,

$$\mathbb{E}\left[S\left(\mathcal{X}\right) \middle| X_1, s_1 \right] \approx \frac{\sum_{m=1}^M S(\mathcal{X}_m)}{M}.$$

I believe Monte Carlo should result asymptotically in a normal distribution, but this resembles a log-normal distribution. How would I diagnose this issue? How should I change my simulation strategy?

I've proved the sum converges. The proof boils down to the unconditional mean of $$X_n <0$$, and sum inside the exponential goes to negative infinity faster than one-half the unconditional variance.

The instability occurs when the sum inside the exponential terms is greater than 0 for a few periods (the process is persistent). Even though it will eventually converge to -$$\infty$$, and can blow up temporarily.

• I don't understand the current simulation scheme, 12000 random paths of what? And what do you mean by a run of monte Carlo? Commented Mar 8, 2023 at 22:12
• @hipHop What's the distribution of the $X_i$ here? Are they identically distributed? Are they independent or dependent? How do you know the sum converges? Commented Mar 9, 2023 at 0:50
• @Glen_b updated Commented Mar 9, 2023 at 17:58
• @svendvn updated Commented Mar 9, 2023 at 17:58
• @hipHopMetropolisHastings, I still don't fully understand what you are simulating. You write that you are simulating 1200*12000 realizations S({X_t}_{t=1}^N), but for which values of N? And how does one run differ from another run? Commented Mar 14, 2023 at 21:04

I am still not sure I understand what you are trying to simulate, but below I have tried to answer the question to how I think it works.

### Simulation setup

The simulations depends on three quantities:

• $$K$$, the number of simulations of the $$(X_1, s_1)$$. That is $$(x_1,\tilde s_1)\sim P_{X_1, s_1}$$
• $$M$$, the number of simulations of $$S_N(X)|(X_1,s_1)=(x_1,\tilde s_1)$$ per simulated value $$(x_1,\tilde s_1)$$.
• $$N$$, the length of the chain $$X_1, \dots, X_N$$.

And results in $$K$$ realizations of the conditional expectation (as a random variable), $$E[S_N(X)|X_1,s_1]$$. Let $$\hat P_{K,M,N}$$ denote the actual distribution of the simulations and let $$P$$ be the hypothesized asymptotic distribution. Let $$d$$ be a favorite distance measure between distributions. The goal of simulations is to determine if $$d(\hat P_{K,M,N},P)$$ can be made infinitesimally small by increasing $$K,M$$ and $$N$$. That can be a daunting task, so let's break it down.

### Size of $$K$$

For a given size of $$K$$, it is known how similar the simulated distribution should be to the desired asymptotic distribution. One can simply simulate from the desired asymptotic distribution $$K$$ times, obtaining a empirical distribution $$\hat P_K$$ and compute $$d(\hat P_K, P)$$.

### Size of $$M$$

The size of $$M$$ should be so big that the variance of $$S_N(X)|(X_1,s_1)$$ becomes negligible. I don't know if it actually has a finite variance, but if one could get some bound on it, $$L$$, it would be known that the variance of $$\hat E_M[S_N(X)|(X_1,s_1)]$$ is less than $$L/M$$. Then, for a given $$K$$ and $$M$$, we can simulate $$K$$ realizations of the desired asymptotic distribution and add $$N(0,L/M)$$ noise, obtaining an empirical distribution $$\hat P_{K,M}$$ and then compute $$d(\hat P_{K,M}, P)$$.

### Size of $$N$$

For a given $$K$$ and $$M$$, I suggest that simulations are used for increasing $$N$$ to show that $$d(\hat P_{K,M,N}, P)$$ can be made as small as $$d(\hat P_{K,M},P)$$. The simulations for different values of $$N$$ doesn't need to be independent to make a convincing argument, so one can simply continue the same $$K\cdot M$$ chains.

In practice one may want to simulate a distribution for $$d(\hat P_{K,M}, P)$$ instead of just a single number. $$K$$ and $$M$$ should be chosen such that $$d(\hat P_{K,M},P)$$ becomes so small that it is convincing when it is shown that $$d(\hat P_{K,M,N}, P)$$ is of similar size.

• Thank you for your thoughtful answer. I believe the issue is the conditional moment I am trying to simulate is either infinite, or the variance is so large that its infeasible to simulate. IE, there are some sample paths with very small probability weight where the expectations goes to infinity. Hence why I see the far right tail. I'm marking your answer as correct, because it addressed my question which I think was ill-posed. Commented May 11, 2023 at 22:53