Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
When we think of linear regression, the implicit assumption is that we only observe a small fraction of a possibly infinite large population.
Thinking of simple averages, imagine a fair die. The expected value of this die is 3.5. This is the population average, because we actually observe the entire population in a sense. My question is the following: how and when can we distinguish between sample averages and population expected values?
You correctly computed the expected value of the random variable that gives the result of the fair die: 3.5. This tells you that if you roll the dice infinitely many times and take the mean of all the individual results, then you will get 3.5. An estimate of this quantity is given by the sample mean obtained by rolling the die a finite number of times and by computing the mean of the individual results.
As an illustration, let's roll the die 10 times
set.seed(1234)
n <- 10
x <- sample(x = 1:6, size = n, replace = TRUE, prob = rep(1, 6))
> x
[1] 2 5 5 5 1 5 2 3 5 5
and compute the sample mean:
mean(x)
[1] 3.8
Let's now roll the die many many times:
set.seed(1234)
n <- 10000
x <- sample(x = 1:6, size = n, replace = TRUE, prob = rep(1, 6))
> mean(x)
[1] 3.5171
