How to use the MCMC method for multivariate distributions? I wish to find the posterior of a joint distribution of 4 parameters whose prior and likelihoods are known, but I do not understand how to accept and reject samples, in any other case other than single variable cases used in Metropolis-Hastings.
I am a mathematics undergrad, and I would be really grateful if you could direct me to some good literature.
Thanking you in advance.
PS:- This is for a problem in Operations Research(Reliability Theory). If you require a more specific description of the problem statement, please let me know.
 A: The Metropolis-Hastings algorithm is applicable whatever the dimension of $\theta$. Finding the posterior of the joint distribution of $\theta_1$, $\theta_2$, $\theta_3$, and $\theta_4$, is equivalent to finding the posterior distribution of $\Theta$, where $\Theta$ is a $4 \times 1$ vector with $\Theta = [\theta_1 \ \theta_2 \ \theta_3 \ \theta_4]$.
Let's apply it to the formalism of the Wikipedia article on the algorithm. Then the most recent value sampled is $x_t = [\theta_{1,t} \ \theta_{2,t} \ \theta_{3,t} \ \theta_{4,t}]$. We next draw a new proposal state $x' = [\theta_1' \ \theta_2' \ \theta_3' \ \theta_4']$ with probability density $p(x'|x_t)$, which is the probability of having $[\theta_1' \ \theta_2' \ \theta_3' \ \theta_4']$ (i.e. their joint probability) knowing $x_t$. 
A common and practical assumption is to assume that parameters are independent, which means that $p(x'|x_t) = p(\theta_1'|\theta_{1,t})p(\theta_2'|\theta_{2,t})p(\theta_3'|\theta_{3,t})p(\theta_4'|\theta_{4,t})$.
A nice example is the following paper : Bird, A. D., Wall, M. J., & Richardson, M. J. (2016). Bayesian inference of synaptic quantal parameters from correlated vesicle release. Frontiers in computational neuroscience, 10, 116.
They compute the posterior joint distribution of having six parameters using the Metropolis-Hastings algorithm. They even provide the JULIA and MATLAB code they used, which is available in the "Supplementary Material" section. 
Hope this helps !
A: 
... you could direct me to some good literature

I found Chib, Greenberg (1995) insightful for understanding the MH algorithm.

but I do not understand how to accept and reject samples

One quick and dirty solutoin is: sample one variable at a time, conditioning on every other variable (and data). i.e., sample $\theta_1 | \theta_{2:4}, x$, then $\theta_2 | \theta_1, \theta_{3:4}, x$, ..., and $\theta_4 | \theta_{1:3}, x$. That constitutes to one sample from the posterior. That is if you want $1000$ samples from the posterior, you would repeat the above procedure $1000$ times.
This is sometimes called Metropolised Gibbs, but as the paper suggests, it is really just MH. You should also be wary of potential modes of failure.
