Concepts of Probability all messed up

Question

I am having a really hard time taking concepts of probability space, experiment, random variables and stitching them together to make a robust understanding of the probability theory.

So every probability theory starts with an experiment. According to Wikipedia "In probability theory, an experiment or trial is any procedure that can be infinitely repeated and has a well-defined set of possible outcomes, known as the sample space".

Now, lets say, our experiment is to go to the library and see the first book that was issued that day. This can either be done just for a day (although of no significance from the point of statistical analysis) or few days. Suppose we go to the library for N days. These repetitions of the experiment are called trials, and these trials make up a composed experiment (or experiment). People sometimes use the term trials and experiment interchangeably or composed experiment and experiment interchangeably. But it's better to think of individuals trials within a single experiment (trials != experiment and composed experiment== experiment).

Now for the above experiment all the books in the library is our sample space. The random variables can be

1. No. of pages in the book (Let's say $X_{1}$)
2. Price of the book ($X_{2}$)
3. No. of authors ($X_{3}$)
4. No. of times the book has been issued before ($X_{4}$)

Now comes the concept of probability distribution of these random variables. Does these distributions and the entire Probability space always come from the experiment or even before we perform the experiment we have a theoretical of these variable ?

On our very first trial , let's say, we find that "The elements of Statistical Learning" is the book issued that day.

Let's assume

$$X_1 = \text{426 pages} $$ $$X_2 = \text{30 dollars} $$ $$X_3 = \text{3 authors} $$ $$X_4 = \text{Issued 50 times before} $$

With every other trial of our experiment we are changing out Probability Space (Sample Space, Set of Events, Probability measure)

So the previous definitions of Random Variables falls flat. Because now that we have a new sample space and Random Variables map outcomes in Sample Space to real numbers, we need new definitions of our random variables every time we perform one more trial. Or we could have definitions which are not influenced by these changes such as

1. No. of pages in the latest book ($Y_{1}$)
2. Price of the latest book ($Y_{2}$)
3. No. of authors of the latest book($Y_{3}$)
4. No. of times the latest book has been issued before ($Y_{4}$)

After performing $N$ trials we will end up with

$$Y_{1} = (y_{11}, y_{12}, y_{13}, \ldots , y_{1N})$$

$$Y_{2} = (y_{21}, y_{22}, y_{23}, \ldots , y_{2N})$$

$$Y_{3} = (y_{31}, y_{32}, y_{33}, \ldots , y_{3N})$$

$$Y_{4} = (y_{41}, y_{42}, y_{43}, \ldots , y_{4N})$$

The probability distributions of these random variables can now be used for statistical testing against true distributions. But again I am confused here, do these true distributions exist ? When one speaks of probability distributions of random variables is he talking about distributions calculate through $N$ trials or even before the experiment is done, that is, while defining probability space at the very beginning do we have a true theoretical probability distribution.

Now comes the question of Independent and Identically distributed random variables. The above random variables ($Y_{i}$) can be assumed as independent (debatable) but I cannot see how random variables here can be identically distributed. They dont even have the same range, Pages can take value from 0-3000 (let's say) , authors can never have such high numbers. So how can we ever assume that random variables are identically distributed ?

Speaking of Central Limit Theorem, it asks us to sum independent and identically distributed random variables but how can we add two different things, to me it seems like adding apples to oranges, No. of pages + Price of the book, doesn't make any sense ? Or are they asking us to add the elements of a single Random Variable($y_{i1}+y{i2}+y_{i3}+y_{i4}+\cdots+ y_{iN}$) but this still feels inconsistent to the definition of CLT which talks about sum of Independent and Identically Distributed Random variables.

One major reasons these doubts stem is because many books use some terms interchangeably like Random variables as a function from sample space to real space and Instances of Random Variables are both termed as Random variables, some never differentiate Probability Space from Real Space, events from random variables etc. Also the conventional dice throw and coin toss examples are way too simple and I feel do a pretty bad job of explaining these concepts and extrapolating them to real world examples. I would really appreciate if someone could make these concepts clearer to me using the example that I used above.

Answer 1

I am afraid you misunderstood many different things and you should study this more carefully.

Using your example, you have four random variables $Y_1,Y_2,Y_3,Y_4$ (for number of pages, price, etc.). The variables are obviously not independent and not identically distributed, what you already noticed. Taking as example number of pages, this variable is defined in terms of probability distribution that describes probabilities of observing any possible number of pages in any possible book. The distribution is a function that maps number of pages to probability of observing book with such number of pages in your experiment. Drawing more samples form the distributions (more trials) does not change a thing about those random variables, you don't need "new definitions". In your example you observed $N$ draws from each of the four random variables. You ended up with observing $N\times 4$ random variables, e.g. $Y_{11}$ in your example is number of pages of the book observed on the first day etc. The $Y_{11}, Y_{21}, \dots, Y_{N1}$ random variables are independent and identically distributed. The $Y_{i1}$ is a random variable because you observed a random book on the $i$-th day, while the observed outcome $y_{i1}$ is not random, it is a known value. We use the observed outcomes to learn something about the unknown and unobservable random variables, this is where we use statistics. I can't see any difference between $Y_1$ and $X_1$ in your description, unless you use $X_1$ to denote any possible book, and $Y_1$ to denote some subset of $X_1$.

To read more check also the following threads: What is meant by a "random variable"?, Can anyone clarify the concept of a "sum of random variables", and What intuitive explanation is there for the central limit theorem?.