I'm a math Ph.D. and I've recently been doing a fair amount of Reinforcement Learning as part of my job. Out of general interest, I would like to read up on the theory of Markov decision processes (MDPs) and partially observable Markov decision processes (POMDPs). Can anyone recommend any books that go through this in depth and with full rigor? Applications aren't important, but I would like to see the book cover whatever is possible also in the continuous-time case and not just the discrete.
There seems to be a few books that fit the description for MDPs, but don't cover POMDPs. Does anyone have any good recommendation(s)?