In [Casella, Berger] Statistical Inference there is a short discussion on countability of sample spaces and its implications:
This distinction between countable and uncountable sample spaces is important only in that it dictates the way in which probabilities can be assigned. For the most part, this causes no problems, although the mathematical treatment of the situations is different. On a philosophical level, it might be argued that there can only be countable sample spaces, since measurements cannot be made with infinite accuracy. (A sample space consisting of, say, all ten-digit numbers is a countable sample space). While in practice this is true, probabilistic and statistical methods associated with uncountable sample spaces are, in general, less cumbersome than those for countable sample spaces, and provide a close approximation to the true (countable) situation.
I'm having a bit of trouble wrapping my head around the nuances of the distinction, and concretely picturing the implications discussed. A couple of questions: how is the way that probabilities are assigned different in an uncountable space? Can you provide an example of the way probabilities are assigned to some uncountable space and how it differs from assignment to the countable space that it is closely approximating? Further, can you provide an example of how the mathematical treatments are different? What makes the uncountable spaces less cumbersome to work with (I would imagine it would be the opposite)? Lastly, I'm not too sure what they mean by measurements of infinite accuracy in this context. While of course the finite set of ten-digit numbers is of finite accuracy and we can certainly work with it, why would a countably infinite be any more philosophically sound to work with? Wouldn't it also be of infinite accuracy (I'm thinking of e.g. the natural numbers, although again maybe I am misunderstanding what is meant by accuracy)?
I imagine some of these could be answered as I read on, but at the very least I'd like to have something in my mind so I know what to pay attention to as I keep reading. I also haven't been able to find much other discussion online regarding the nuances of countability, so any insights or references that I missed would be greatly appreciated.