Regression's population parameters

Question

Suppose I've specified a linear regression model: $$ Y = \beta_0 + \beta_1 X + \epsilon $$ where $\beta_0$, $\beta_1$ are the population parameters. My question is: why are these parameters populational? My intuition is that they are calculated based on all the possible realizations of the random variables $X$ and $Y$. The coefficient $\beta_1$ is $E(XX')^1E(XY)$ where the expected values ​​are calculated based on all the possible realizations of both variables and their probabilities.

Answer 1

The linear regression model that you specified can be view as a linear approximation of expected value of $Y$ on $X$. In many presentation about regression this fact is not clearly affirmed but it seem me very relevant. So, as usual in mathematical statistics, you have some parameters $\theta$s that characterize the joint distribution $(Y,X)$. These are population parameters or, equivalently, the exact/true parameters. You can see $\beta$s as transformations of $\theta$s, so you can see the $\beta$s as population parameters also.

Changing point of view we can think about statistical inference. So, very briefly, we can collect data from $Y$ and $X$ and estimate a regression in order to analyze some associations of interest (note: estimated approximation of expected value of $Y$ given $X$). Here you achieve the estimated parameters $\beta_{est}$ that seem me what you have in mind (est stand for estimator). Moreover you can interested in many features of $\beta_{est}$, for instance efficiency. In fact $\beta_{est}$ have its variance while $\beta$ is a constant. This happen precisely because $\beta$ collect population parameters and $\beta_{est}$ their estimators.

Answer 2

As I understand it, a main point of this question concerns the relevance of the term "population." In much of statistics, and particularly in regression, that term either needs to be dropped altogether, or at the very least, placed in quotes. It leads to silliness and misunderstanding, and is often is just plain wrong.

For regression, the problem with the "population" term lies in the definition of $E(Y | X = x)$, which is what the regression model is supposed to aim at. In the classic finite population model, there is simply not enough data (sometimes no data at all) in cohorts defined by $X=x$ for the population average in the cohort to have any useful meaning. Instead, it is at best a noisy estimate of some true quantity. The problem is greatly exacerbated in the case where $X$ is vector, i.e., in multiple regression, because the cohorts defined by multiple fixed values of $X$ greatly dwindle in sample size.

An example: Let $Y$ be a person's height, and let $X$ be the numerical (ordinal) value of one of their 16-digit cards. (Credit card, etc.). In the population of everyone on the planet at this instant who own such a card, there will be a certain number of people (very small) whose number is 3422337799818871. In the population setting, the value $E(Y | X=3422337799818871)$ refers to the average height of these people. This number may be an average of just one or two person's heights, clearly a very noisy estimate. The true regression function, in the "population" sense, is then these $E(Y | X=x)$ as a function of $x=0,1,2,\dots,9999999999999999$. It is a very noisy, wobbly function.

This "population regression function" is clearly very different from what we understand as the true regression function in this case, which is correctly given by $E(Y | X = x) = \beta_0 + \beta_1 x$, where $\beta_1 = 0$. (Although I'd like to hear an argument for why $\beta_1$ might be different from 0!)

So, rather than refer to populations, the regression model should instead refer to potentially observable data, which applies equally to the population as well as any sample. (The population itself is comprised of particular realizations of potentially observable data.) This definition of the regression model in terms of potentially observable data seems to be implicit in the OP's correct understanding that the regression model should refer to "possible realizations" rather than "populations."