I have fairly good practical experience with Metropolis-Hastings and Gibbs sampling, but I want to get a better mathematical understanding of these algorithms. What are some good textbooks or articles that prove the correctness of these samplers (more algorithms would also be great)?
For a handbook and an extensive coverage, the following one is very moderately priced.
Robert and Casella (2010) have a good deal of theory.
I'm not sure whether this is exactly what you're after, but a couple of articles I've found useful on theoretical properties of various Metropolis-Hastings algorithms are:
The book by Robert & Casella (mentioned above) is a very good and thorough resource, but you may also find these two of use:
These both also have information on Gibbs sampling. I suppose you may also find some other information on Markov Chains useful. I generally use:
But there is also some good information in most MCMC books on this.