Understanding Expected Value of observed responses in Regression I have a quick question about
the following is a discussion on the online Penn State Regression methods course:

Recall that, in fitting a regression model to data, we attempt to
  estimate the average—or expected value—of the observed responses
  $E(y_i)$ at any given predictor value x. That is, $E(y_i)$ is the
  population regression function. Because the average of the observed
  responses depends on the value of x, we might also denote this
  population average or population regression function as $\mu_{Y|x}$
Now, if there is no bias in the predicted responses, then the average
  of the observed responses $E(y_i)$ and the average of the predicted
  responses $E(\hat y_i)$ both equal the thing we are trying to
  estimate, namely the average of the responses in the population
  $μ_{Y|x}$. On the other hand, if there is bias in the predicted
  responses, then $E(y_i)$ = $μ_{Y|x}$ and $E(\hat y_i)$ do not equal
  each other. The difference between $E(y_i)$ and $E(\hat y_i)$ is the
  bias $B_i$ in the predicted response, i.e
$B_i$=$E(y_i)−E(\hat y_i)$

(See "Bias in predicted responses" section here for context).
My question here is about the average/expected value of observed responses, $E(y_i)$ that we're talking about. Could someone intuitively explain to me what this means? How can observed y-values in a sample have a "true" population value, or am I looking at it the wrong way?
 A: In the comment below, you write:

"We attempt to estimate the average/expected value of observed
  responses". What do they mean by this? As in, I'm trying to
  intuitively understand what the "Average" of observed responses would
  be?

This concept of "average" really clicked with me when I started modeling things explicitly in the Stan language (a statistical language for Bayesian inference). I'm going to be stealing some of my notation from their manual.
So, we can usually write a linear regression like this:
$y_i = \beta_0 + \beta_1x_i + \epsilon_i$ where $\epsilon_i \sim N(0, \sigma^2)$
A few things to note: $y$ is the observed dependent variable, $x$ is the observed independent variable, and $i$ represents that each individual has their own value for it. $\beta_0$ is the intercept and $\beta_1$ is the slope for $x$. $\epsilon$ is the residual term. We can see that we assume that the residuals are normally distributed with a mean of zero and some variance.
Now, this can be rewritten as:
$y_i - (\beta_0 + \beta_1x_i) \sim N(0, \sigma^2)$
Which we can further simplify to:
$y_i \sim N(\beta_0 + \beta_1x_i, \sigma^2)$
And lastly, since $\hat{y} = \beta_0 + \beta_1x_i$, we can simplify again:
$y_i \sim N(\hat{y_i}, \sigma^2)$
Notice the subscript $i$. This means that, for every individual, we say their observed score comes from a normal distribution with a mean of the predicted value and a given residual variance. Notice that, for every person, this predicted value is different, but the variance is the same. The variance being the same is where we get the assumption of homogeneity of variance.
Now, I don't really like using "average value" and "expected value" interchangeably, because for some distributions, they are not the same thing. But for the normal distribution, the average and expected values are the same (because the mean is equal to the mode of a normal distribution). But that is what they mean by average. You are basically saying:
"Every observation of y is distributed with a mean of y-hat and a variance of sigma squared." This only clicked for me when I started using Stan, because in that language, you can literally code it as:
y ~ normal(beta[0] + beta[1] * x, sigma)

