I am studying MCMC with "Pattern Recognition and Machine Learning" Book by Christopher Bishop. In the chapter of MCMC, this book introduces Markov Chain also a little bit.
However, while reading the book, I wonder why Markov chain is needed. Because to conduct the Metropolis-Hasting algorithm, for every step I make sample from proposal distribution and then decide whether it is accepted or not.
For me, this Metropolis-Hasting algorithm is more like rejection sampling. Adopting proposal distribution that we can directly draw sample from and deciding the sample should be accepted or not are similar.
Where is the room for using Markov chain? Do I need to calculate 'transition matrix' for Metropolis-Hasting algorithm? At this my confusing state, I feel that I can conduct Metropolis-Hasting sampling without need of transition matrix of Markov Chain.
I am very confusing now. This book just says "under some circumstances a Markov chain converges to the desired distribution". But it does not say how can I design the Markov chain to converge distribution I desire. And many of the materials in the Internet also seems to skip this part. I thought designing transition matrix of MC is important part to conduct MCMC. But now, I guess it is not necessary.
Thanks in advance.