What is the relationship between the function $\mathbb{E}(Y \mid X = x)$ and linear regression?

Question

Consider the function

$$ r(x) = \mathbb{E}(Y \mid X = x) $$

This has been called the regression function in a textbook I'm using. I'm trying to figure out the relationship between this function and the classical linear regression model.

So, I know that it is a theorem* that we may write

$$ Y = r(X) + \epsilon $$

for some random variable $\epsilon$ s.t. $\mathbb{E}(\epsilon) = 0$.

Now suppose that we have

$$ Y = \beta_0 + \beta_1 X + \epsilon $$

This is the classical 1-dimensional regression function (assuming the $\beta_0$ and $\beta_1$ minimize the residual sum of squares).

Question: Is it then a mathematical theorem that if $Y$ is defined as above, that

$$ r(X) = \mathbb{E}(Y \mid X) = (\beta_0 + \beta_1 X)? $$

And is this why the function $\mathbb{E}(Y \mid X)$ is called the "regression function"?

EDIT: The theorem that I am making use of is as follows (from All of Statistics pg. 89):

Regression models are sometimes written as

$$ Y = r(X) + \epsilon $$

where $\mathbb{E}(\epsilon) = 0$. We can always rewrite a regression model this way. To see this, define $\epsilon = Y - r(X)$ and hence $Y = Y + r(X) - r(X) = r(X) + \epsilon$. Moreover, $\mathbb{E}(\epsilon) = \mathbb{E}\mathbb{E}(\epsilon \mid X) = \mathbb{E}(\mathbb{E}(Y - r(X)) \mid X) = \mathbb{E}(\mathbb{E} ( Y \mid X) - r(X)) = \mathbb{E}(r(X) - r(X)) = 0$.

Answer 1

Summarizing the question:

Given $Y = \beta_0 + \beta_1 X + \varepsilon$, is it then a mathematical theorem that $r(X) = \mathbb{E}(Y \mid X) = (\beta_0 + \beta_1 X)$?

Yes, by basic properties of expectation:

$$ \begin{align} \operatorname{E}(Y\mid X) & = \operatorname{E}(\beta_0 + \beta_1 X + \varepsilon) \\[6pt] & = \operatorname{E}(\beta_0) + \operatorname{E}(\beta_1 X) + \operatorname{E}(\varepsilon) & & \text{(linearity of expectation)} \\[6pt] & = \beta_0 + \beta_1 X + 0 & & \text{(Noting that $X$ is constant here} \\[-2pt] & & & \quad \text{because we conditioned on it.)} \\[6pt] & = \beta_0 + \beta_1 X \end{align} $$

The historical reasons for regression being called regression relate to Galton noticing the "regression to the mean" effect -- initially in an experiment in plants involving seed-size of offspring compared to the seed size of parents. A relationship through the mean seed size on both variables will have slope less than $1$ (which slope can be estimated by what we call linear regression). The smaller the slope the stronger the "regression" effect. The issue is illustrated by Galton in the linked pdf by heights of children (as adults) compared to average heights of parents (females being scaled up by a constant factor of $8\%$ to make them comparable to males). The diagrams on the third to fifth pages indicate something of what was observed.

So an attempt to estimate the size of this "regression to the mean" is obtained by what came to be called linear regression. Of course there's nothing special going on - the regression to the mean isn't some special biological "drive to mediocrity" as might originally have been guessed, but a fairly simple consequence of the mathematics of the situation in essentially the same sense that correlations are always between $-1$ and $1$.