Basic question about probability distributions and sampling I am currently reading up on limit theorems and laws of large numbers and there is one thing in particular that I find confusing and this is the notion of the “true” mean of a given population. To make things more concrete, let’s say we have a school of 1000 students and we want to find the mean height of those students. Now, what is the distribution of the heights of those students? Is it just a probability mass function wherein the argument takes on discrete values (namely the height of every student) and the PMF is uniform (a student is picked randomly)? Or is it some unknown continuous distrbution of heights that we can only infer using statistics? In other words, in the context of limit theorems, what does the “true” mean or population refer to in this case? Is it just the “center of gravity” so to speak of the PMF corresponding to the heights of the students? Or is it something more abstract and unknown? And shouldn’t the distribution necessarily be discrete in this case given the discrete nature of the sample space at hand (namely the students in the school)?
Thank you in advance.
 A: Sampling theory often involves analysis of problems where you take a random sample of size $n$ from a population of size $N$.  The latter can either be a finite integer (finite population) or infinity (infinite superpopulation).  If you are sampling without replacement then you cannot have more sample values than population values so you must have $n \leqslant N$.  If you want to examine a limit where $n \rightarrow \infty$ then the quantity of interest must be defined for all $n \in \mathbb{N}$, which can only be the case if $N = \infty$ (i.e., for an infinite superpopulation).
So you are right to be concerned here.  It does not make sense to look at a limit of a quantity as $n \rightarrow \infty$ in the context of a problem with a finite population.  You say you are reading about this but you don't give any quotation or link to your material so it is not clear where this comes up.
A: If the height of the $i$-th student is seen as a random variable $X_i$ and if we assume that all the random variables $X_i$ have the same theoretical distribution, then the "true" mean is simply the mean of this distribution, i.e. the theoretical mean of each $X_i$. 
In practice, you only see realizations of these random variables, i.e. real values $x_i$ (the observed heights) and limit theorems are useful when the number $N$ of students is so large that you can't observe all of them (e.g. not simply the students of a school but of a whole country) and they tell you that the sample mean is "a good approximation" of the true mean when the size of the sample is large enough. 
