In [Casella, Berger] Statistical Inference there is a short discussion on countability of sample spaces and its implications:

This distinction between countable and uncountable sample spaces is important only in that it dictates the way in which probabilities can be assigned. For the most part, this causes no problems, although the mathematical treatment of the situations is different. On a philosophical level, it might be argued that there can only be countable sample spaces, since measurements cannot be made with infinite accuracy. (A sample space consisting of, say, all ten-digit numbers is a countable sample space). While in practice this is true, probabilistic and statistical methods associated with uncountable sample spaces are, in general, less cumbersome than those for countable sample spaces, and provide a close approximation to the true (countable) situation.

I'm having a bit of trouble wrapping my head around the nuances of the distinction, and concretely picturing the implications discussed. A couple of questions: how is the way that probabilities are assigned different in an uncountable space? Can you provide an example of the way probabilities are assigned to some uncountable space and how it differs from assignment to the countable space that it is closely approximating? Further, can you provide an example of how the mathematical treatments are different? What makes the uncountable spaces less cumbersome to work with (I would imagine it would be the opposite)? Lastly, I'm not too sure what they mean by measurements of infinite accuracy in this context. While of course the finite set of ten-digit numbers is of finite accuracy and we can certainly work with it, why would a countably infinite be any more philosophically sound to work with? Wouldn't it also be of infinite accuracy (I'm thinking of e.g. the natural numbers, although again maybe I am misunderstanding what is meant by accuracy)?

I imagine some of these could be answered as I read on, but at the very least I'd like to have something in my mind so I know what to pay attention to as I keep reading. I also haven't been able to find much other discussion online regarding the nuances of countability, so any insights or references that I missed would be greatly appreciated.

  • $\begingroup$ Many of your questions have been answered in other threads: visit the hits in this site search for "countability paradox". If you have remaining questions, please feel free to edit this post to focus it on one of them. $\endgroup$
    – whuber
    May 18 at 18:54
  • $\begingroup$ @Shivashriganesh Mahato. Perhaps you might consider working through pp56-74 of the following article, Mauldin, D. (1987) Tutorial on probability, measure and the law of large numbers. Los Alamos Science No. 15. The article was a special edition dedicated to the mathematician Stanislaw Ulam. $\endgroup$
    – microhaus
    May 18 at 19:34
  • $\begingroup$ It gives a beautifully clear, concise summary of the various aspects of the development of modern probability theory from Laplace to Kolmogorov, and may be useful for the discussion of going from countable to uncountable sample spaces you have asked about. The exposition only relies on intuitive ideas and elementary tools, with a taster of why measure theory was developed, and throughout relies on showing how we can think of these developments through an extended coin flipping example. $\endgroup$
    – microhaus
    May 18 at 19:35
  • $\begingroup$ But where I think the article excels the most is that the narrative uses various seminal paradoxes, such as Bertrand's paradox and the Banach-Tarski paradox to motivate what made the journey to the development of the formalism difficult, and at the same time gives you a flavour of the deep conceptual debates going on within mathematics at the time. No intermediate stats or probability textbook I know of provides an account with this kind of sensitivity to the history of mathematical ideas. $\endgroup$
    – microhaus
    May 18 at 19:35