1
$\begingroup$

When we think of linear regression, the implicit assumption is that we only observe a small fraction of a possibly infinite large population.

Thinking of simple averages, imagine a fair die. The expected value of this die is 3.5. This is the population average, because we actually observe the entire population in a sense. My question is the following: how and when can we distinguish between sample averages and population expected values?

$\endgroup$
  • 1
    $\begingroup$ Well, more generally we assume a data-generating process as if the observations were sampled from an infinitely large population. With your die example, you might just say that symmetry & background knowledge make a certain distribution a plausible model for the data-generating process without reference to a population. But I don't understand your question: if you know what a sample average is, & what an expected value is, what more do you need to distinguish them? $\endgroup$ – Scortchi Dec 15 '15 at 15:44
  • $\begingroup$ Let me rephrase my question: can we say that the sample average is an estimate of the expected value? $\endgroup$ – ChinG Dec 15 '15 at 16:01
  • 1
    $\begingroup$ If the distribution has an expectation (see Why does the Cauchy distribution have no mean?) then you can certainly say the sample mean is an estimator of it. But you can say that of any sample statistic: the question's how an good estimator it is & whether you can find a better one. So then you have to decide what defines the goodness of an estimator. An example of the sample mean's not being the best estimator of the expectation by any reasonable criterion is when $X$ has a uniform distribution between $0$ & unknown $\theta$. $\endgroup$ – Scortchi Dec 15 '15 at 16:46
3
$\begingroup$

You correctly computed the expected value of the random variable that gives the result of the fair die: 3.5. This tells you that if you roll the dice infinitely many times and take the mean of all the individual results, then you will get 3.5. An estimate of this quantity is given by the sample mean obtained by rolling the die a finite number of times and by computing the mean of the individual results.


As an illustration, let's roll the die 10 times

set.seed(1234)
n <- 10
x <- sample(x = 1:6, size = n, replace = TRUE, prob = rep(1, 6))
> x
 [1] 2 5 5 5 1 5 2 3 5 5

and compute the sample mean:

mean(x)
[1] 3.8

Let's now roll the die many many times:

set.seed(1234)
n <- 10000
x <- sample(x = 1:6, size = n, replace = TRUE, prob = rep(1, 6))
> mean(x)
 [1] 3.5171
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.