# Population, sample and model

Background:

In Statisical Methods by Pfaffenberger and Patterson, they say that

"A parameter is a numerical measure of a population characteristic." (p. 306) "A statistic is a numerical measure calculated from a set of sample observations."

"Sampling error is the difference between the value of the parameter in the population and the value of the statistic in the sample, excluding other erros, such as $\ldots$" (p. 311)

They also define the population mean and the sample mean to be respectively: $$\mu=\frac{\sum_{i=1}^N x_i}{N}$$ (p. 39) and $$\bar{x}=\frac{\sum_{i=1}^n x_i}{n}$$ (p. 39), where N is the number of individuals in the population and $x_1,\ldots,x_N$ are the measurements of each of the individuals in the population and $x_1,\ldots,x_n$ is a sample of size $n$ of the $N$ measurements. The population standard deviation and sample variance are defined respectively as $$\sigma^2=\frac{\sum_{i=1}^{N}(x_i-\mu)^2}{N}$$ (p.47) and $$s^2=\frac{\sum_{i=1}^{n}(x_i-\bar{x})^2}{n-1}$$ (p. 47) where the elements $x_i$ are defined as for the mean. They also make some examples where the definitions are used. In example 8.1 on page 309, they calculate $\mu=\frac{x_1+x_2+x_3+x_4+x_5}{5}=\frac{7.5+8.0+2.2+15.9+16.4}{5}=10.0$. Then they list the different subsamples possible with three elements. One possible sample is $x_1,x_2,x_3$. Then $\bar{x}=5.90$. The sampling error is then 10-5.9=4.10.

Definitions:

Random variable: mapping from the sample space to the real line. If the RV $X$ is discrete, then $\text{Pr}(X=x)=\text{Pr}\{s_j\in S\mid X(s_j)=x\}$, where $S=\{s_j\}$ is the sample space.

Realization: a particular value of the random variable $X$. The random variable takes the different values $x_i$ according to a probability distribution. These probabilities are the long run frequencies of the different outcomes in the experiment, i.e $\text{Pr}(X=x_i)=\text{lim}_{n\rightarrow \infty}\frac{\sum I(X_j=x_i)}{n}$ (assuming $X$ is discrete)

Experiment: you are considering the result of some action in the future, ie. the outcome of an action that have not occured yet. This action can result in different events. One of these events must occur. Performing this action is called performing an "experiment" in statistics.

Distribution: the long run relative frequency of different outcomes for a discrete RV. I am not sure of the interpretation in the continuous case. Here the distribution is a density, so that $\text{Pr}(X\in (a,b])=\int_a^b f(t)\text{d}t$

Population: all of the individuals that you consider modelling with the random variable. So if you model the wage of the employees at a firm with a random variable, then all the employees are the population.

Questions re-edited:

1) What are the authors assuming when they say that the mean is equal to the mean of all the individuals in the population, that $\text{Var}(\frac{X_1+\ldots+X_N}{n})=0$?

2) Normally we would assume that $X_1,\ldots,X_N$ distributed as $X$ and say that $\bar{X}$ is an estimator of $\mu=\text{E}(X)$. Then some say we assume an infinite population. Why is this so? Where is the proof of this?

Original questions:

1) According to Weak law of large numbers in finite populations, the set consisting of all the realisations of the random variables corresponding to the individuals in the population is still a sample. Why then do Pfaffenger and Patterson say that the population mean is exactly the mean of the realisations of the RVs in the sample?

2) If $\mu=\frac{x_1+\ldots+x_N}{N}$, then the distribution of the realisations of the RVs obtained in the experiment must be the true distribution of the underlyding random variable?

3) Is the distribution of the RVs in the sample the true distribution of the population characteristic? Or do we say that the values obtained are only realisations of a RV, where the mean of the random variable may not coincide with the mean of the sample.

4) How can you tell someone that the mean of all the employees is not the true mean (i.e sample mean of all the employees may not be equal to the expectation of the population RV)? This is fine when you only look at a sample, but I look at all employees.

5) We may choose to model the population RV with a parametric distribution, then it makes sense that $\bar{x}\neq \mu$. However, is it also a model assumption that I can view the different salaries as realisations of a random variable, whatever be the distribution? Is this the reason the mean of the salary RV does not necessarily equal the mean of all the observations in the population, even if the true distribution of the random variable was available?

• Your previous question and this set of questions suggest you understand at least some of the terms "random variable," "realization," "experiment," "distribution," and "population" in unconventional senses. These questions are puzzling because they use the words in ways that don't quite make sense. So that your post can be answered, could you please (1) narrow it to a single question and (2) define, as clearly as you can, the technical terms that you are using? – whuber Aug 29 '16 at 17:08
• To me this is calling for a community wiki type of answer, if it does not exist already. – luchonacho Aug 29 '16 at 17:39
• @whuber thanks for your answer. Could you please tell me in which way(s) I use the terms wrong? I have added definitions to the terms you have listed. Next I will shorten down the questions to a single one. – FredrikAa Aug 29 '16 at 18:24
• It's hard to tell. I'm just reacting to difficulties making sense of some combinations of words, like "mean of the realisations of the RVs in the sample." I suspect you may have a good understanding of most of these words, but the meaning of phrase itself is obscure. – whuber Aug 29 '16 at 19:38

1. I would agree with the definition you've encountered in the WLLN rather than their definition of a "population mean" as being a "sample mean of the entire population". They differ in some important ways. Suppose for instance that I conduct a survey which samples the first 100 people who come through the mall on a Saturday. My experiment does not generalize to all mall patrons, or even the first 100 to come through on any other day of the week. An important distinction in frequentist statistics is that of an infinitely sized "population". In my mall survey, the "population" would be defined as an infinite number of independent replications of my experiment in which I "rewound time" and did my survey again and again and again. This is counterfactual reasoning. If you sample 100% of a finite population without replacement and obtain a sample mean $\bar{X}$ you can still replicate that experiment an infinite number of times and sample people again and again and again obtaining sample mean $\bar{X}$ an infinite number of times. So a proper finite sample CI would be $\bar{X}$ to $\bar{X}$.

2. The answer above obviates this.

3. The population characteristic does not have a distribution. By "values obtained" do you mean the estimand? In statistics, if a sample is obtained from IID observations with a mean $\mu$, then the sample average will also have mean $\mu$, that's simply a result of linear operators.

4. Practically, it would not be an estimate if you sampled everybody. You can safely say that you have found the "true mean".

5. RVs are mappings or functions, they have nothing to do with sample/population distinctions. I think what you are saying here is a mistake. A parameter in a model may be the mean, and we may call that $\mu$ (or $\theta$ or... there is no conventional notation here), but there are other distributions with different parameters like shape, scale, shift, or rate parameters which we may call $\mu$ or $\theta$ or... Obviously if you call a "scale" parameter $\mu$ then no the sample mean does NOT estimate this value. However, if the mean is a well defined quantity, it is some function of the parameters and you will consistently estimate this value with the sample mean.

• I'm interested in your fourth point. Why would it not be an estimate? What about the concept of superpopulation? (you might census all population in a country but there is still the potential superpopulation in that country) What about measurement errors? (surely you want to separate them from underlying variables) – luchonacho Aug 29 '16 at 18:31
• thanks for your reply @AdamO. I have some questions to your answer: (3) why do not RVs have a probability distribution? Does not the histogram you draw from a very large amount of samples approximate the probability distribution of the RV? (4) how do you sample everyone in an infinite population? So every finite collection of results of individuals will be a sample? Here is the part where I am falling off: the population is indefinitely large, so it will be larger than every number you can imagine. – FredrikAa Aug 29 '16 at 18:43
• Does this thinking hold for discrete RVs as well? Do you know places in the literature where this is discussed? (1) What type of CI have you listed? – FredrikAa Aug 29 '16 at 18:43
• So all finite "populations" are considered samples in statistics? – FredrikAa Aug 29 '16 at 18:53
• @luchonacho The WLLN applies to infinite populations and the Pfaffberger definition of a population mean does not. The OP has not wrapped his head around a super population, the idea of infinite independent replications of a study experiment. – AdamO Aug 29 '16 at 20:16