I'm attempting to build a hierarchical Bayesian model. For various reasons (including my own edification), I want to do this from scratch (i.e., without using the various packages and libraries written for this purpose). However, all the examples and tutorials I've come across make use of Stan, JAGS, PyMC3, etc. Does anyone know of or have an example of a working script that implements a simple hierarchical Bayesian model? Preferably it would be written in either python, R, or Matlab.
1 Answer
Colin Carol has a nice blog series on doing Hamiltonian Monte Carlo (the method by which Stan and PyMC3 do the sampling) from scratch.
It's really good, and he even has a github repo called minimcmc
which does some sampling. If you are interested in MCMC/HMC, I would start there.
If you want to avoid MCMC/HMC all together, you are really limiting yourself. Gelman has an example of the beta binomial hierarchical model in chapter 5 of Bayesian Data Analysis. I managed to implement that model here both analytically and with pyMC3. I think these are some good resources to get you started.