# Techniques for improving mixing when sampling from a multidimensional posterior

I'm currently trying to use the Metropolis-Hastings algorithm to sample from a posterior distribution of the form

$$p(\theta | y ) \propto \prod_{ij} \phi (\theta_{ij}) \times \prod_{i=1}^n \pi_{y_i}(x_i|\theta)$$

Where $$\phi()$$ is the standard normal density function. This posterior is the result of taking the multinomial logit model with multivariate normal prior $$N_p (0, I_p)$$.

I have taken the proposal distribution to be

$$q(\theta_{new} | \theta_{old}) \sim N_p (\theta_{old}, I_p)$$

As far as I can tell my implementation of the algorithm is correct, but mixing is incredibly poor. The acceptance ratio is consistently on the order of $$1\times 10^{-3}$$ or worse.

What are some techniques for improving mixing for sampling from a high dimensional posterior? If the dimension was low, I would plot the distribution to get an idea of what proposal distribution would suit it. But when the posterior is high dimensional, this doesn't seem feasible.

The bad news is that we would need to know more about the model you are trying to fit. For example, it might make sense to block the $$\theta$$ in various ways, do Gibbs sampling, various HMC or other gradient based techniques etc etc.