0
$\begingroup$

I aim to achieve variance reduction in Random Walk Metropolis Hastings algorithm by introducing stratification to the random walk jumps. What I have tried is to make use of Latin Hypercube Sampling in each iteration.

lhs_1 <- randomLHS(1, 5)
ym_1 <- qnorm(lhs_1, sd = sqrt(5))

In above code I predefine the Random Walk jumps by using Latin Hypercube sampler. I have five groups in stratification. Values in ym_1 can serve as different proposals. I can try each of these proposals and pick the one with the highest density. This naturally increases the acceptance ratio but chain explores a limited region.

I am not sure about how to make use of these proposals. More generally, I am not sure about how to correctly achieve variance reduction here by using stratification in Random Walk Jumps. Any idea would be greatly appreciated.

Thanks.

$\endgroup$
3
$\begingroup$

The Latin hypercube code you wrote may not be accomplishing what you want.

This code draws 5 variables with marginals on U(0,1) with no stratification. You will not achieve any variance reduction this way:

lhs_1 <- randomLHS(1, 5)
ym_1 <- qnorm(lhs_1, sd = sqrt(5))

If you want one variable with 5 strata across [0,1]:

lhs_2 <- randomLHS(5, 1)
ym_2 <- qnorm(lhs_2, sd = sqrt(5))

In most other applications, the way to achieve variance reduction with a Latin hypercube is to draw the whole Latin hypercube at once, not iteratively as one does in a Metropolis Hastings algorithm. If you can determine a way to have a lot of Latin hypercube samples available up front, and then use them as the algorithm progresses, you might be able to make this work. There is also the technique of Progressive Latin hypercube sampling you might want to explore here

$\endgroup$
3
  • $\begingroup$ Thank you Mr. Carnell! Inside Metropolis-Hastings algorithm I can create the LHS matrix beforehand and use these values as proposals. I can iterate through that matrix in a loop and run qnorm(lhs_matrix[i,], mean=chain[i,], sd=sd) in order to get the proposal value for chain[i+1,]. But in that case I can have 5 different proposals, each pointing different direction. I can calculate the best direction by getting density values of each and decide to go to that direction, but it makes chain perform badly in terms of estimations. Do you have any suggestions about how to achieve VRF using LHS here? $\endgroup$
    – boyaronur
    Apr 21 at 13:18
  • 1
    $\begingroup$ I've put some thought into it, and tried to find other researchers using LHS for MCMC, but I haven't found anything on point. I'm worried that the theory of Metropolis-Hastings requires a sample from the full distribution, and doesn't allow for stratification in sampling. On the other hand, a LHS ought to help in specifying the posterior in some way, but I don't have an answer on how. $\endgroup$
    – R Carnell
    Apr 21 at 14:16
  • $\begingroup$ Thanks a lot for investing your time on it! There are types of Metropolis Hastings algorithm with multi try proposals. I am trying to find a way to apply stratification using that. $\endgroup$
    – boyaronur
    Apr 22 at 7:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.