# How to Understand the Relationships Among Random Variables, Samples, and Populations?

I am trying to understand the relationship of a Random Variable to a population and random samples. I understand that a Random Variable is a function that maps each event in a sample space to a number. In this context, what would be a sample space and mapping? Would each sample be governed by the probability mass function?

Do we think of each sample as a Random Variable that takes on values from a population (sample space) randomly?

Or do we think of a population as a Random Variable and our sample as a random collection of those points?

I would appreciate if you can explain this to someone who is new to mathematical statistics. Thank you!

• Please have a look at the following link might be useful – Vendetta Jul 20 '16 at 5:40

Between the question and the existing answers there is some contradiction and confusion. Let's start by clearing up the definitions. Then I will provide an intuitive analog for thinking about them. I will finish by sketching out how a sample really doesn't have to be thought of any differently than a single element of a population.

### Definitions

A population is usually modeled as a set $$\mathcal S$$ together with a probability measure $$\mathbb{P}$$ on that set. The probability measure indeed is a set function: there are distinguished subsets of $$\mathcal S$$, called "events," to which are assigned numbers between $$0$$ and $$1$$, inclusive. When that assignment satisfies simple and intuitively obvious axioms, it models probabilities.

A (univariate) random variable $$X:\mathcal{S}\to\mathbb{R}$$ assigns numbers to the elements of $$\mathcal S$$. It does so in a way that is consistent with the events of $$\mathcal{S}$$ and the subsets of $$\mathcal{R}$$ of the form $$(-\infty, a]$$: that is, every set of the form $$E=\{s\in \mathcal{S}\mid X(s) \le a\}$$ must be an event. This enables us to construct the distribution of $$X$$ defined by $$F_X(a) = \mathbb{P}(X \le a).$$It is defined for all real numbers $$a$$ and has values ranging from $$0$$ through $$1$$.

A multivariate random variable is defined similarly, but takes $$m$$ separate values: its codomain is $$\mathbb{R}^m$$ rather than $$\mathbb{R}$$, for $$m\ge 1$$.

Sampling is modeled in various ways. One procedure is to construe a sample as a finite sequence $$(s_1, s_2, \ldots, s_n)\in \mathcal{S}^n$$. It is implicit that subsets of the form $$E_1\times E_2\times \cdots \times E_n \subset \mathcal{S}^n$$ are events. Another procedure focuses on a random variable, rather than the population, and views an independent and identically distributed ("iid") sample as a sequence $$X_1, X_2, \ldots, X_n$$ of random variables on $$\mathcal S$$ (a) that are independent and (b) for which all the $$X_i$$ have the same distribution.

### A conceptual model

There is an intuitive, rigorous physical model of these mathematical abstractions in which

• The population is represented by slips of paper ("tickets") in a box, with one or more identically labeled slips representing each $$s\in\mathcal{S}$$;

• The probability of an event is the proportion of all tickets it comprises in the box; and

• A random variable $$X$$ consists of writing a sequence of $$m$$ numbers on each kind of ticket.

In this model, a random variable is realized by mixing the tickets in the box, blindly drawing one out, observing the values of $$X$$ written on it, and returning the drawn ticket to the box (so that the contents of the box remain unchanged). From this point of view a sample is obtained through some definite procedure for drawing one or more tickets. An iid sample is simply obtained as $$n$$ realizations.

See https://stats.stackexchange.com/a/54894/919 for an elaboration of these ideas.

Although this conceptual model seems to suppose the population is finite, the entire theory of probability--including for "infinite populations"--has been developed with it. See Edward Nelson, Radically Elementary Probability Theory (Princeton University Press, 1987).

### How to model and think about samples

This setup does not look sufficiently rich or general to handle important statistical settings like time series analysis, financial analysis, sampling without replacement, etc. However, various constructions show just how powerful it is. I will give one (important) example to illustrate.

All possible realizations of a sample of size $$n$$ consist of sequences $$\mathbf{s}=(s_1, s_2, \ldots, s_n)$$ with $$s_i\in\mathcal S$$. The set of all such realizations is, by definition, the Cartesian product $$\mathcal{S}^n$$. The events of $$\mathcal S$$ determine events of the form $$E = E_1\times E_2 \times \cdots \times E_n\subset \mathcal{S}^n$$ where each $$E_i$$ is an event of $$\mathcal S$$. Specifically, $$\mathbf{s}\in E$$ if and only if each $$s_i\in E_i$$. (These "basis" events in turn define a larger set of events of $$\mathcal{S}^n$$, according to the axioms of probability, but the details of that need not concern us here.) This makes it possible to define probability measures on $$\mathcal{S}^n$$ that are "compatible" with $$\mathbb{P}$$.

Any sequence of random variables becomes a single (multivariate) random variable defined on $$\mathcal{S}^n$$ in the simplest way possible: create a ticket to represent every possible $$\mathbf{s}=(s_1,s_2,\ldots, s_n)$$ and on those tickets write down, in order, the numbers specified by $$X_1(s_1)$$, then $$X_2(s_2)$$, and so on. This is how multivariate random variables usually arise (and explains why we need to accommodate them in the theory).

This Cartesian product construction thereby enables us to view any realization of a sample of $$\mathcal{S}$$ as a realization of one element of the population $$\mathcal{S}^n$$, provided we are willing to work with multivariate random variables instead of just univariate ones. This is a huge conceptual simplification, for now statistical inference can largely be framed as the study of drawing a single ticket out of a (much larger) box. Moreover, by selecting a suitable probability function for the events of $$\mathcal{S}^n$$ we may model complex sampling procedures (giving rise to theories of sampling and experimental design) as well as interdependence among the random variables (giving rise to analyses of time series, panel data, spatial phenomena, financial series, and more).

### Conclusions

In summary, comparing the definitions given here to those in the question will reveal where the question might have gone astray and will settle most of the issues it raises. And consider both what has been included here and what has not been mentioned:

• Although measurement can be modeled (and usually is) with tickets in a box and random variables, it is not an essential part of any of the concepts.

• "Randomness" arises only through the concept of drawing tickets from a box. It is not a part of any of the mathematical constructions or definitions.

• Sampling, even the most complex designs, is modeled mathematically. Samples can be analyzed as mathematical objects.

• Uniform distributions are not an essential part of the concept of sampling.

A 'random variable' is a mathematical object, more precisely a measurable function.

A 'random' sample is not a mathematical object. It is only a way to pick individuals at random in a population. A way to do it is to simply consider that the population is uniformly distributed, so every individuals have the same chance to get picked, and then to pick the number of individuals you want to form your sample. This way of sampling data doesn't involve considering that sample are random variables.

• I am not sure I am clear. How would you explain that the expected value of the average of a collection of random variables from the same distribution is the same as the expected value of the individual random variables? – verkter Jul 19 '16 at 6:23
• Let's say we have two independent variables X and Y following a Bernoulli's distribution of parameter p. Then mean(X)=mean(Y)=p, and mean(1/2*X + 1/2*Y)=1/2*mean(X) + 1/2*mean(Y)=p. This result can be generalized to the case you refer in your comment. It does work that way because X and Y are random variables, which are well-defined mathematical objects that work that way. What is usually done in statistics is that we attach a random variable to a sample. Then if you pick at random this sample, the sample's 'value' will be the outcome of the random variable attached to it. – Syzygy Jul 19 '16 at 6:45
• So there are two ways of thinking about this. First way is theoretical and second way is practical. I am trying to understand how practical sampling relates to the theory. – verkter Jul 19 '16 at 6:55

Neither a sample nor a population can be considered variables. I think measurement is the missing piece of the puzzle in your question.

Strictly speaking, a sample is considered random when every element in the population has an equal chance of being sampled (simple random sampling) or at least a known probability (random sampling in a broader sense).

The "randomness" in random variable has more to do with the fact that you're measuring things that vary and that you can't know in advance what their value will be.

Now for the sample space, for tossing a coin it is just {heads, tails}, but when you are measuring a person's height for instance, the sample space is all the heights a human being could possibly have (sample space of infinite size theoretically -- even though not practically), no matter what your population (or sample) size is. Edit I realize there is room for debate on that specific point and some prefer considering the sample space as the height of all individuals in the population.

• If I understand you correctly, a random variable and a random sample are related? Just trying to understand where and how theory meets practice. – verkter Jul 20 '16 at 4:47
• No they aren't related. The way you carry out your sampling has no bearing on the "random" property of the variable you are measuring - you assume it follows some theoretical distribution because of its alleged nature, which is not affected by the sampling method. – Dominic Comtois Jul 20 '16 at 5:10