Optimal software package for bayesian analysis I was wondering which software statistical package do you guys recommend for performing Bayesian Inference.
For example, I know that you can run openBUGS or winBUGS as standalones or you can also call them from R. But R also has several of its own packages (MCMCPack, BACCO) which can do bayesian analysis.
Does anybody have any suggestions as to which bayesian statistics package in R is best or about other alternatives (Matlab or Mathematica?)
The main features I am looking to compare are performance, ease of use, stability and flexibility
 A: Within the 3 BUGS variants (openBUGS/winBUGS, jags) jags seems to be the most promissing as for the future feature development, and openBUGS/winBUGS seem to be dead projects. However, jags is still lacking some niceties present in openBUGS/winBUGS (also look here). On the other hand, jags has removed some limitations present in WinBUGS, e.g.:
x ~ dnorm(0, tau) 
tau ~ dgamma(1.0E-3, 1.0E-3) # in WinBUGS, you cannot do this, 1.0E-3 is too small 
                             # for dgamma (use e.g. dgamma(0.01, 0.01))

The good news is that with most models, you can run them in all the 3 tools with just minimal changes, so you can switch to different tool later without much problems (that's what I do).
However, for some reasons (e.g. lack of parallelism and interpreter nature), it is not true that these BUGS variants are fastest way to do bayesian analysis! In fact, quite the opposite. BUGS projects are good to test and develop complicated models on small datasets. Once you have the model developed, and need to run it repeatedly on large datasets, it is more efficient to use different tools. 
For example the CppBugs / rcpp combo is said to be 5-10x faster than BUGS variants. The principle is that you basically compile your model into a C++ program, which runs much faster. Also have a look at Dirk Eddelbuettel's blog for Rcpp test - looks brutally fast. You can also play with parallelism.
You can also do parallel computation in WinBUGS using bugsparallel.
A: External BUGS variants are the standard. Working within R may be convenient, but I'd be surprised if those packages are as mature and perform as well. Using a library which bridges R and the external program is usually the most common compromise.
I use the jags/rjags combo (jags might be roughly considered a dialect of bugs). I haven't tried the other bugs variants, but the reports I've heard are that jags's performance and ability to deal with numerical issues is a bit better than the other bugs variants. I find jags easy to use, but of course, you need some knowledge of bayesian data analysis to know how to use it.
