Background:
In Statisical Methods by Pfaffenberger and Patterson, they say that
"A parameter is a numerical measure of a population characteristic." (p. 306) "A statistic is a numerical measure calculated from a set of sample observations."
"Sampling error is the difference between the value of the parameter in the population and the value of the statistic in the sample, excluding other erros, such as $\ldots$" (p. 311)
They also define the population mean and the sample mean to be respectively: $$\mu=\frac{\sum_{i=1}^N x_i}{N}$$ (p. 39) and $$\bar{x}=\frac{\sum_{i=1}^n x_i}{n}$$ (p. 39), where N is the number of individuals in the population and $x_1,\ldots,x_N$ are the measurements of each of the individuals in the population and $x_1,\ldots,x_n$ is a sample of size $n$ of the $N$ measurements. The population standard deviation and sample variance are defined respectively as $$ \sigma^2=\frac{\sum_{i=1}^{N}(x_i-\mu)^2}{N} $$ (p.47) and $$ s^2=\frac{\sum_{i=1}^{n}(x_i-\bar{x})^2}{n-1} $$ (p. 47) where the elements $x_i$ are defined as for the mean. They also make some examples where the definitions are used. In example 8.1 on page 309, they calculate $\mu=\frac{x_1+x_2+x_3+x_4+x_5}{5}=\frac{7.5+8.0+2.2+15.9+16.4}{5}=10.0$. Then they list the different subsamples possible with three elements. One possible sample is $x_1,x_2,x_3$. Then $\bar{x}=5.90$. The sampling error is then 10-5.9=4.10.
Definitions:
Random variable: mapping from the sample space to the real line. If the RV $X$ is discrete, then $\text{Pr}(X=x)=\text{Pr}\{s_j\in S\mid X(s_j)=x\}$, where $S=\{s_j\}$ is the sample space.
Realization: a particular value of the random variable $X$. The random variable takes the different values $x_i$ according to a probability distribution. These probabilities are the long run frequencies of the different outcomes in the experiment, i.e $\text{Pr}(X=x_i)=\text{lim}_{n\rightarrow \infty}\frac{\sum I(X_j=x_i)}{n}$ (assuming $X$ is discrete)
Experiment: you are considering the result of some action in the future, ie. the outcome of an action that have not occured yet. This action can result in different events. One of these events must occur. Performing this action is called performing an "experiment" in statistics.
Distribution: the long run relative frequency of different outcomes for a discrete RV. I am not sure of the interpretation in the continuous case. Here the distribution is a density, so that $\text{Pr}(X\in (a,b])=\int_a^b f(t)\text{d}t$
Population: all of the individuals that you consider modelling with the random variable. So if you model the wage of the employees at a firm with a random variable, then all the employees are the population.
Questions re-edited:
1) What are the authors assuming when they say that the mean is equal to the mean of all the individuals in the population, that $\text{Var}(\frac{X_1+\ldots+X_N}{n})=0$?
2) Normally we would assume that $X_1,\ldots,X_N$ distributed as $X$ and say that $\bar{X}$ is an estimator of $\mu=\text{E}(X)$. Then some say we assume an infinite population. Why is this so? Where is the proof of this?
Original questions:
1) According to Weak law of large numbers in finite populations, the set consisting of all the realisations of the random variables corresponding to the individuals in the population is still a sample. Why then do Pfaffenger and Patterson say that the population mean is exactly the mean of the realisations of the RVs in the sample?
2) If $\mu=\frac{x_1+\ldots+x_N}{N}$, then the distribution of the realisations of the RVs obtained in the experiment must be the true distribution of the underlyding random variable?
3) Is the distribution of the RVs in the sample the true distribution of the population characteristic? Or do we say that the values obtained are only realisations of a RV, where the mean of the random variable may not coincide with the mean of the sample.
4) How can you tell someone that the mean of all the employees is not the true mean (i.e sample mean of all the employees may not be equal to the expectation of the population RV)? This is fine when you only look at a sample, but I look at all employees.
5) We may choose to model the population RV with a parametric distribution, then it makes sense that $\bar{x}\neq \mu$. However, is it also a model assumption that I can view the different salaries as realisations of a random variable, whatever be the distribution? Is this the reason the mean of the salary RV does not necessarily equal the mean of all the observations in the population, even if the true distribution of the random variable was available?
