I am trying to understand the relation between population/probability space/sampling. My arguments are divided in 3 sub-questions which trace my attempt to link in a logical way the three concepts. I am using an Economic example, but I regard my question as generic.

Consider a target population of individuals; for each individual in this population we believe that $$income=beta*education+u$$

where $income, education, u$ are real numbers. $u$ collects all the variables affecting income in addition to education.

We are interested in learning about $\beta$.

Sub-question 1: Imagine to extract at random an individual $m$ from the population and observe her income, education, and additional features (denoted by the random variables $Y_m, X_m, U_m$). Intuitively, I understand why we can say that the income, education, additional features of the extracted individual are random variables: we don't know a priori which individual will be picked from the urn representing the whole population and, hence, we attach a probability to each potential outcome. More formally, to define a random variable we need a probability space $(\Omega, \mathcal{F}, Pr)$. Is $\Omega$ set equal to the population?

Sub-question 2: If the answer to question 1 is YES, then, if we knew the entire population (i.e., if we knew $(\Omega, \mathcal{F}, Pr)$) and suppose $E(X_m^2)\neq 0$ and $E(X_mU_m)=0$, we could easily compute the exact value of $\beta$ as $$ \beta=E(Y_mX_m)/E(X_m^2) $$ Correct?

Sub-question 3: The problem is that we don't know the entire population (i.e., we don't know $(\Omega, \mathcal{F}, Pr)$) and so we try to approximate in some good way $E(Y_mX_m)$ and $E(X_m^2)$ by appropriately taking a subset of the whole population (sampling). For example, a way to appropriately take a subset of the whole population is the following: for $m=1,...,M$:

  • We draw at random an individual from the urn containing the entire population, we label him/her with the index $m$, and we register his/her income and education level (denoted by the random variables $Y_m, X_m)$. The additional features affecting income remain unobserved (denoted by the random variables $U_m$).

  • We put back in the urn individual $m$.

The sampling scheme just described implies that $$ (i) \hspace{1cm}\{Y_m, X_m, U_m\}_{m=1}^M \text{ are i.i.d. across $m$} $$

We then define $$ \hat{\beta}=\frac{\frac{1}{M}\sum_{m=1}^MY_mX_m}{\frac{1}{M}\sum_{m=1}^M X_m^2} $$ By $(i)$, $\frac{1}{M}\sum_{m=1}^MY_mX_m\rightarrow_p E(Y_mX_m)$ and $\frac{1}{M}\sum_{m=1}^M X_m^2\rightarrow_p E(X^2_m)$.

Hence, $$ \hat{\beta}\rightarrow_p \beta $$ Correct?

Sub-question 4: Suppose also that $U_m$ is a continuous random variable. Does this imply stating that the population is very large or a continuum or infinite?

  • 1
    $\begingroup$ Question says "in Econometrics". Clearly your examples are economic, but in what sense are you imagining or implying that the stipulation "in Econometrics" might affect possible answers? Is it the same question if you cut that out? (This is to try to help other people, as I don't envisage answering myself.) $\endgroup$
    – Nick Cox
    Commented Mar 1, 2018 at 12:46
  • $\begingroup$ Yes, it is the same. $\endgroup$
    – Star
    Commented Mar 1, 2018 at 12:50
  • 1
    $\begingroup$ So, why not clarify by cutting "in Econometrics" and underlining that although you use examples from economics for concreteness you regard the questions as generic? $\endgroup$
    – Nick Cox
    Commented Mar 1, 2018 at 13:05

2 Answers 2


Sub-question 1. Yes, this is how you define the population from the probabilistic point of view. But it usually does not coinside with the physical population (e.g. all adult citizens of USA). For example, long with John Smith, a real electrical engineer from NY, this population also includes his possible alter ego - an alternative John Smith who moved to Texas as a boy, recieved a different education, and now works in a bar. And this alternative John is unobservable, but he may have a non-zero probability, whatever it means.

Sub-question 2. Yes, this is the only value of $\beta$ that does not contradict your assumptions. However, the assumption that $X$ and $U$ are uncorrelated is rather heroic. In practice, some components of $U$ may depend on education or even affect it, and this is what econometrists "in real life" struggle with.

Sub-question 3. This is totally correct from theoretical point of view. But you may still argue whether your samples are really i.i.d.. When you make a survey of income in 2017, you observe only one world of many possible. E.g. you see income of John Smith and Jane Doe in case of Trump's victory and Brexit, but you know that Clinton's victory would affect both their incomes, therefore they are not independent. Thus, you need to assume a kind of ergodicity - that a sample from a single universe has the same properties as a sample from multiple universes.

Sub-question 4. Surely yes. If you assumed sampling from a finite population, $U$ would have only a finite number of outcomes and thus would be discrete.

If you imagine multiple alternative universes, there may well be a continuum of them (even if there is a small number of people living in each universe). Technically, income is still discrete, because money is discrete: you don't usually report your income as 123456 dollars 76.8(8) cents. But a continuous distribution may approximate all substantial properties of $U$'s distribution and be much easier to manipulate.

  • $\begingroup$ Thanks. Regarding sub-question 4: I am a bit confused also reading the answer below. Suppose that I assume not only that $U_m$ is a continuous random variable, but also that it has a STRICTLY positive pdf on the entire real line. Does your conclusion change? $\endgroup$
    – Star
    Commented Mar 1, 2018 at 18:50
  • $\begingroup$ No, it doesn't change. The most popular distribution is normal, which indeed has pdf strictly positive everywhere, and it is frequently used to approximate discrete distributions, e.g. binomial. $\endgroup$
    – David Dale
    Commented Mar 1, 2018 at 19:40
  • $\begingroup$ Thanks. Last points I would like to clarify before accepting your answer: let's imagine that I sample from a FINITE population without replacement; (1) in this case the i.i.d. hypothesis fails and I cannot apply L.L.N. to show consistency. Actually, in that case, consistency as usually meant does not make a lot of sense. Correct? (2) If, instead, I sample from a finite population with replacement, I'm back to i.i.d., LLN, and usual notion of consistency. Correct? $\endgroup$
    – Star
    Commented Mar 5, 2018 at 16:55
  • $\begingroup$ (1) I suspect there is a modification of LLN for the case of "weakly dependent" observations. However, asking for consistency means "increase sample size indefinitely and look at the distribution of your estimator", and with sampling from finite population w/o replacement it is plainly impossible. Thus, notion of consistency is inapplicable here. (2) Exactly so! $\endgroup$
    – David Dale
    Commented Mar 5, 2018 at 17:04

Good answer from @David Dale there. Following on from his statement on Sub-question 4 I'm confused how you expect the hypothetical properties of Um would impact any statement you would make about the population. The population size would define the upper limit of granularity for Um that could be achieved in practice, whatever its .

It is also affected, as @david dale suggests, by the inherent granularity of your Y value, salaries. In a large population is likely to have more of an impact on how continuous it truely is. In a working population of 100,000,000, with median income of 100,000 it would be likely that people close to median will share their salary to the same cent with at least 10 other people.

In any case, I can't see how Um's properties would backwash the other way.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.