Why describe a sample as i.i.d.?

Question

I’m hoping for some help understanding the concept of RVs with respect to their use in the theory of drawing inferences on a population from a sample.

To draw inferences on a population using a sample it is said that the observations must be i.i.d. RV's. I'm considering the example of a weighted die. If you wanted to test if a die was weighted you could buy one and roll it 1,000 times. On each roll you could record the number generated in order to understand the probability distribution of the die. The outcome from rolling a die once can be represented as a RV that maps from the outcome of the roll to an integer in the set $\{1,2,3,4,5,6\}$.

I know that a RV $X=f(x)$ is a real-valued function that maps from its domain to a subset of the real numbers so if I were to describe this experiment before it took place I may write down the vector $[X_{1},X_{2},…,X_{1000}]$ where $X_{i} \,\, \forall \,\, i\in[1,1000]$ are i.i.d. RVs. We can imagine rolling the die 1,000 times and then writing in the results for each experiment, yielding $[x_{1},x_{2},…,x_{1000}]$, the realizations of the RVs.

Although $[X_{1},…,X_{1000}]$ are identical they seem to be regarded in texts as distinct RVs, and I'm wondering why? If we recognize that $X_{i}=f(x) \,\, \forall \,\, i$ (i.e. each $X_{i}$ is the exact same function), and the reason that $X_{i}$ need not equal $X_{j} $ for $i\neq j$ is because we fed a different input into $f(x)$, then isn’t it equally valid to say that $X_{1},X_{2},…,X_{1000}$ represent multiple observations on the exact same RV?

Indeed, if you have two random variables $f(x)$ and $g(x)$, but $f(x)=g(x)$ and they have identical domains and identical ranges, then it seems confusing to argue that $f(x)$ and $g(x)$ are “different”? What seems to actually be happening is that you have one RV, $f(x)$, which is a way of describing the possible outcomes of rolling a dice, and on each roll you input a different element from the domain into the function and hence get a distinct output. So can anyone explain to me the intuition for describing this process as i.i.d. RV's, as opposed to different observations from the same RV?

Answer 1

You have to recall a random variable is just a function that maps an event space into a probability space. For a single realization from a single observation, it may seem redundant to consider that such mappings are defined similarly over $n=1000$ replications. However, the statistical experiment is based on some summary measure or "data reduction" defined on the event space and probability space. The fact that IID has reduced these concepts to mere cartesian products of the basic observation is a product of the stringent IID assumption.

Designating each random variable allows you to formalize the event space, define estimators and calculate their distribution, and set up probability models for outcomes. In many experiments, $X_1, X_2, \ldots, X_n$ are neither independent nor are they identically distributed, such as with Urn models. So you can represent the probability as the product of conditional probabilities for each of $X_1$, $X_2 | X_1$, $X_3 | X_2, X_1$, etc. Indeed many useful limit theorems can be derived in the presence of mildly correlated observations and or distributional differences such as the general Lyapunov or Lindeberg-Feller Central Limit Theorem.