Strategies to re-write the slow part of Metropolis Hastings in Rcpp I want to speed up my R implementation of a Metropolis Hasting procedure by replacing the slow parts with functions written in Rcpp.
There are already some examples online using Rcpp to speed up Gibbs sampling and Metropolis Hastings.
However, these examples implement the entire sampler in Rcpp. In contrast, I hope to rewrite only the slower part of my R implementation so that I can keep the niceties of R as much as possible. (I'm trying to avoid premature optimization here.)
However, profiling my code does not reveal any particularly candidate for optimization. Most of the code already uses R's vectorized operation & built-in random number generator.
What parts of the Metropolis Hastings algorithm can be rewritten in Rcpp to achieve a speed up? Or do I indeed have to rewrite the entire loop in Rcpp itself?
On why coding the Metropolis Hastings by hand: One of my parameter is a binary matrix of 0s and 1s, and my understanding is that Stan / JAGS don't accommodate such kind of variable.
On why this is a statistics question: I already know how to use Rcpp or to do code profiling, so I'm not limited by programming ability. The question requires understanding MCMC procedures, especially which part can be ripped out and put in Rcpp, and which part cannot. That understanding is a lot more about statistics than programming.
P/S: I can't answer this question myself partially because I don't understand why R is slower than C++ / Rcpp, and thus can't reason through which part of Metropolis-Hastings would be faster in Rcpp. If the answer touches on the reason why a certain part is better written in Rcpp, it'd be very instructive.
 A: To elaborate on my comment about loops, here is a benchmark of looping in R and looping in Cpp. I construct a function in both that sums up numbers upto p. The Rcpp file:
// [[Rcpp::depends(RcppArmadillo)]]
// [[Rcpp::export]]
double summingC(double p)
{
  double add_sum = 0;

  for(double i = 1; i <= p; ++i)
  {
    add_sum = add_sum + i;
  }
 return(add_sum);
}

The R code and benchmarking:
set.seed(100)

library(Rcpp)
library(rbenchmark)

# Loading Rcpp file
sourceCpp("summing.cpp")

add_sum <- 0
summingR <- function(p)
{
for(i in 1:p)
{
    add_sum <- add_sum + i
}
return(add_sum)
}

## benchmarking between Cpp and R
benchmark(summingC(1e7), summingR(1e7))

And the output below indicates that even for doing a simple addition, Cpp is 15 times faster.
> benchmark(summingC(1e7), summingR(1e7))
             test replications elapsed relative user.self sys.self user.child
1 summingC(1e+07)          100   1.505     1.00     1.471    0.004          0
2 summingR(1e+07)          100  22.740    15.11    22.654    0.022          0

So it seems best to put all loops in Rcpp.
