# Single Component Metropolis-Hastings (i.e. component-wise updating)

So, let's say I have the following 2-dimensional target distribution that I would like to sample from (a mixture of bivariate normal distributions) -

import numba
import numpy as np
import scipy.stats as stats
import seaborn as sns
import pandas as pd
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt
%matplotlib inline

def targ_dist(x):

target = (stats.multivariate_normal.pdf(x,[0,0],[[1,0],
[0,1]])+stats.multivariate_normal.pdf(x,[-6,-6],[[1,0.9],
[0.9,1]])+stats.multivariate_normal.pdf(x,[4,4],[[1,-0.9],[-0.9,1]]))/3
return target


and the following proposal distribution (a bivariate random walk) -

def T(x,y,sigma):

return stats.multivariate_normal.pdf(y,x,[[sigma**2,0],[0,sigma**2]])


The following is the Metropolis Hastings code for updating the "entire" state in every iteration -

#Initialising

n_iter = 30000

# tuning parameter i.e. variance of proposal distribution
sigma = 2

# initial state
X = stats.uniform.rvs(loc=-5, scale=10, size=2, random_state=None)

# count number of acceptances
accept = 0

# store the samples
MHsamples = np.zeros((n_iter,2))

# MH sampler
for t in range(n_iter):

# proposals
Y = X+stats.norm.rvs(0,sigma,2)

# accept or reject
u = stats.uniform.rvs(loc=0, scale=1, size=1)

# acceptance probability
r = (targ_dist(Y)*T(Y,X,sigma))/(targ_dist(X)*T(X,Y,sigma))
if u < r:
X = Y
accept += 1
MHsamples[t] = X


However, I would like to update "per component" (i.e. component-wise updating) in every iteration. Is there a simple way of doing this?

You should, however, consider the sampling scheme: it's better to have a reversible scheme than just to cycle through $\theta_1,\theta_2,...,\theta_k$ (for example you can randomize the order or you can go forward and then backward within each iteration -- those are examples of reversible schemes).