What do the elements of a sample space in a probability space represent in a real-world statistical test?

Question

Consider the measurable space $(\Omega,\mathcal{F})$ where $\Omega$ is the sample space and $\mathcal{F}$ is the a set of observable events. Further assume that we are estimating a linear regression $GDP_i=\alpha+\beta_1 POP_i+\beta_2 TECH_i+\epsilon$ where the dependent variable is GDP of a country and independent variables are population and some measure of technology.

The question is how do I formalize this problem in measure-theoretic probability theory? More specifically, what would be the sample space, $\Omega$ in this test? (I am thinking along the lines of it being the set of all possible countries in the world, but what exactly are those?)

EXAMPLE

If we are talking about a coin toss experiment where we toss the coins twice, then our sample space is $\{HH,HT,TT,TH\}$. So my question is in modeling GDP in countries, what is our sample space.

Answer 1

Your model is in fact a bit more complicated, it is more like $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \epsilon$ where $\epsilon$ is a normal random variable with mean zero and standard deviation $\sigma$ (and some additional assumptions).

So if we know values for $x_1$ and $x_2$ then we can compute $y=\beta_0 + \beta_1 x_1 + \beta_2 x_2$ which results in a number that we denote $y|_{x_1,x_2}=\beta_0 + \beta_1 x_1 + \beta_2 x_2$ meaning the value of $y$ for given values for $x_1$ and $x_2$.

But, as $y|_{x_1,x_2}$ is a number , that implies that $y|_{x_1,x_2} + \epsilon$ is a normal random variable with mean $y|_{x_1,x_2}$ and standard deviation $\sigma$ or $y|_{x_1,x_2} \sim N(\beta_0 + \beta_1 x_1 + \beta_2 x_2,\sigma)$ and this is your model in terms of random variables.

So GDP given the population and given the technology is a random variable that is distributed as above.

If you look in probability theory you will find that a random variable is a map from the sigma-algebra $\mathcal{F}$ into the real axis.

so if $x_1$ is a value population and $x_2$ is a value for technology then then there are many countries having the same values for $x_1$ and $x_2$ and the GDPs of these countries (with the same values for $x_1$ and $x_2$) are normally distributed. So in $\mathcal{F}$ you have subsects of countries with that value for $x_1$ and $x_2$.

Note that the regression defines not only one random variable, but a random variable for each combination of values for $x_1$ and $x_2$.

In your example with the two coins you can enumerate the sample space because it is finite, in the regression case this is not possible because it is uncountably infinite.

Answer 2

I would start with even simpler model, not much different from the sample with heads and tails you wrote: $$GDP_i=\alpha+\beta POP_i+\varepsilon_i$$ This is a cross-sectional model, like in your problem. Here $i$ - is the country, and GDP and POP are its GDP and population, and $\varepsilon_i$ - are the error terms from bernulli distribution with equal probabilities to go up or down by 1%. Of course, in reality it's going to be something less discrete, maybe more like Binomial distribution, but we're trying to focus on what's essential here.

I'm assuming that you are conducting the obesrvational study, not forecasting. Hence, $POP_i$ - are not random, they're measured and known for each country. Hence, your sample space is really defined by $\varepsilon_i$. Say, you have 3 countries, then the sample space has 8 elements:$\Omega=\{ (-1,-1,-1),(-1,-1,1),(-1,1,-1),\dots,(1,1,1)\}$

Answer 3

In a regression setting, you can say you get a sample of random variables:

$$ (X_1, Y_1), (X_2, Y_2), ..., (X_n, Y_n)$$

where $X_1$ is the technology and the population of country 1 and $Y_1$ is its GDP.

If you think these countries all come from the same distribution, you can say these $(X_i, Y_i)$ pairs all follow the distribution of the random variables $X, Y$. The $X$ and $Y$ variables may or not be related. Your linear regression example established a linear link between $X$ and $Y$, but that is not needed to think about the sample space.

$X, Y$ then are random variables that go from $\Omega$ to the real numbers (or, say, integers in the case of population). You can think of this as saying: "if the state of the world is $\omega_j$, then the population and technology and GDP will be such and such". What are the states of the world that can make country $i$ have population = 100000, technology = 4, GDP = 10000? I'm not sure how to make this concrete (or whether it matters). Maybe we can say it's the combination of all historical events that lead to the present state of the country.

I'm not sure taking a stance on what is an individual $\omega$ matters that much. In fact, the nice thing about talking about random variables is that you can usually work with the realizations (which will be integers or real numbers) and then just focus on thinking whether the countries are independent draws, whether $X$ and $Y$ have a linear relationship and whether, say, $Y$ conditional on $X$ can be described as having a normal distribution. These seem to be easier questions than trying to reason about the underlying $\omega$s.