3
$\begingroup$

I am performing an experiment on Metropolis-Hastings. I have written a proof that returns for stocks have to be a function of the Cauchy distribution and that the distribution of $\beta$ in the equation $x_{t+1}=\beta{x_t}+\epsilon_{t+1},\beta>1$ is also the Cauchy distribution. I chose to seed the random numbers so that the maximum likelihood estimator is $\beta=.99553.$ and $\alpha=.09452$. This violates the assumption $\beta>1$. White in 1958 showed that there was no Likelihoodist solution for this problem as the MLE is the OLS estimator but the test statistic is the Cauchy distribution, so a Bayesian method is being used, hence the Metropolis-Hastings problem.

The data generating function was $x_{t+1}=1.00004x_t+\epsilon_{t+1}$, where $\epsilon\sim\mathcal{N}(0,1.00004^t)$ and $x_0=10$. There is an added restriction that $x_t>0,\forall{t}$.

The likelihood function for this problem is $$\frac{1}{\pi}\frac{\gamma}{\gamma^2+(x_{t+1}-\beta{x_t}-\alpha)^2}.$$ The prior has the following properties:

  1. $\Pr(\beta\le{1})=0$
  2. $\Pr(\gamma\le{0})=0$
  3. Over the region that is reasonable for $\beta$ and $\sigma$, the prior is $1\times{1}\times{1}/(1+\alpha^2)$. The marginal prior is flat over $\beta$ and $\sigma$ in reasonable regions, but not $\alpha$.
  4. In unreasonable regions the prior decays as a half-Cauchy distribution. This only impacts the burn-in as the likelihood of different starting points is at least ten orders of magnitude from the MLE.

This prior creates the problem for me. It is a cliff and the MLE is over the cliff. If $\beta$ were allowed to range freely over the reals, then it would roughly marginalize out as a Cauchy distribution peaked at .99553, ignoring the prior. The true parameter is 1.00004, which is close to the cliff since $\gamma\approx{.65}$.

Where I am having difficulty in my posterior sampling is that the kernel estimates of the posterior fall off fast too far from the cliff, for largish sample sizes. When I do not thin, the problem goes away because I have so many bins, but of course thinning is required as there is autocorrelation in this sample out to 133 lags. I lose a lot of sample from thinning.

I am using the normal distribution as my proposal density, with covariances of zero. Other than letting my computer run for 20 hours to get a massive sample, which will result in thin bins, any suggestions on changes I could make to keep the kernel estimates from falling off too far from $\beta=1$. I am using the density() function in R. I am hoping to avoid programming my own kernel smoothing function.

Improve this question
$\endgroup$
3
  • $\begingroup$ When you mention the restriction $x_t>0$ for all $t's$ does that mean that the normal is truncated, which makes $x_t$ and $\epsilon_t$ dependent? Also, why is the variance a power of t? Is this a typo? And are you sure your posterior distribution is proper, given that the series is not stationary? $\endgroup$
    – Xi'an
    Commented Jan 19, 2017 at 14:58
  • $\begingroup$ Yes, the Cauchy is truncated. The posterior is proper because I use a proper prior, although, of course, there is no mean. Yes, it is to the $t$ power because $\gamma=f(\frac{\sigma_{t+1}}{\sigma_t})$ by mathematical definition and I want the scale and the location to track each other. Otherwise, if $\gamma=1$, but $x_{t}$ grows to 100,000,000 or such, then there is almost no uncertainty, compared to when $x_t=10$. Under certain restrictive assumptions, there is a proof that the posterior with a flat prior is Cauchy. As pointed out in the lit, the restrictions may not hold in practice. $\endgroup$
    – Dave Harris
    Commented Jan 19, 2017 at 20:56
  • $\begingroup$ Sorry, the normal in the data generating function is truncated. I should also have truncated the Cauchy likelihood and properly adjusted the constant of integration from $1/\pi$. $\endgroup$
    – Dave Harris
    Commented Jan 19, 2017 at 21:03

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.