Gibbs sampling for one variable in a row

Question

In the gibbs sampling, if I sample only one variable repetitively (say 3 times) while other variables remain the same, and sample next one variable for 3 times and so on..

$x_{1_1}^* \sim p(x_1^* |x_2,x_3) $, $x_{1_2}^* \sim p(x_1^* |x_2,x_3) $, $x_{1_3}^* \sim p(x_1^* |x_2,x_3) $

$x_{2_1}^* \sim p(x_2^* |x_{1_3}^*,x_3) $, $x_{2_2}^* \sim p(x_2^* |x_{1_3}^*,x_3) $, $x_{2_3}^* \sim p(x_2^* |x_{1_3}^*,x_3)$

• Scan order1 : $(x_1,x_1,x_1, x_2,x_2,x_2, x_3,x_3,x_3)$

Would this Markov chain sample from target distribution as original gibbs sampling?

In addition, would sampling a block of variables in a row be valid as well?

• Scan order2 : $(x_1,x_2,x_3,x_1,x_2,x_3,x_4,x_5,x_6,x_4,x_5,x_6,…)$

In my opinion, no matter how many times the chain samples one variable from proper conditional distribution, it always samples from target distribution, even if samples are highly correlated.

If my thinking is right, is there any algorithm using this kind of scan order?

Answer 1

If you sample $$X_1^t\sim\pi(x_1|X_{-1})$$ ten times in a row, say, and only keep the $10$th draw, the previous $9$ draws are wasted. If you keep all ten draws, the method is formally valid, since each draw keeps the target as its stationary distribution, but the chain will take ten times longer to reach stationarity. There is thus little appeal in using repeated draws from a given conditional.