So I've recently started an introductory course in econometrics and I'm having trouble grasping the idea of Random Variables and Sample distributions

  1. If we have a population and we take a sample $Y= \{Y1, Y2,...,Yn\}$, is $Y$ a random variable? Because what I usually read in texts is that Y is a random variable but I don't get how?
  2. I understand the concept behind the distribution of the sample mean: Basically you do repeated sampling, find the mean of each sample and then draw the curve of the mean i.e. what values it can take with what probabilities. I'm assuming my concept is clear, can anyone confirm?
up vote 4 down vote accepted

The random variable $Y$ describes a relationship between events and the corresponding probabilities of those events. In more practical terms, a random variable describes a data-generating process. When you generate a random data point that is described by the random variable $Y$, the probability distribution of $Y$ describes the probability distribution of values that can result.

You can think of a "population" as an infinite reservoir of values drawn from $Y$. Sampling from a population is analogous to repeatedly drawing new values from $Y$. A sample of size $N$ is a size-$N$ collection of individual draws from $Y$.

The sample is clearly not the same thing as the random variable itself, so we need a different notation for it. Let's call it $s = \{y_1, y_2, \dots, y_N \}$. Each $y_n$ is a single draw from $Y$. The sample mean is a single number. Let's call it $\bar s$. It is the mean of the sequence $s$, i.e. $\bar s = \frac{y_1 + y_2 + \dots + y_N}{N}$.

We can make an interesting observation here! $N$ independent, identical draws from a random variable $Y$ is the same thing as one draw from each of $N$ independent, identical random variables $Y_n$. Now, we can talk about the sample itself as a random variable $S = \{ Y_1, \dots, Y_N \}$.

Note the difference between

$$ s = \{ y_1, \dots, y_N \} $$

and

$$ S = \{ Y_1, \dots, Y_N \} $$

$S$ is random: it is a sequence of random variables. $s$ is not random. It is the realized value of a draw from $S$, i.e. a sequence of realized values of draws from $Y_1, \dots, Y_N$.

Therefore the sample mean itself can be restated as a random variable $\bar S$.

Compare

$$ \bar s = \frac{ y_1 + \cdots + y_N}{N} $$

with

$$ \bar S = \frac{ Y_1 + \cdots + Y_N}{N} $$

$\bar s$ is just a number: it is the mean of a sequence of numbers $y_1, \cdots, y_N$. But $\bar S$ is a random variable! Specifically, it is a statistic, a single quantity that is calculated from a sample. The value of a statistic for a specific sample is a realization of the distribution for that statistic.

Being a random variable, draws from $\bar S$ are described by a probability distribution. The distribution of sample means, across all possible samples, is described by the distribution of $\bar S$. This distribution is the sampling distribution of the mean.

With regard to your first question, you are probably confused between the random variable $Y$ and the matrix $Y$. It is an unfortunate clash in notation that random variables and matrices are both conventionally written with capital letters. It is often mathematically convenient to express samples as matrices, so that you can do linear algebra operations on observed data (to generate estimates from that data, e.g. with ordinary least squares). The matrix $Y$ would be a matrix of observed values. Take care to observe the context, to avoid this confusion.

To address your 2nd question, there are many ways to derive or describe a sampling distribution. One possible technique is called resampling: repeatedly draw samples from a population that is distributed according to $Y$, and measure the sample mean in each of those samples. The distribution of those sample means should follow the sampling distribution of the mean.

  • Hello! Thank you for the detailed answer. However, I've lost you on this statement :"N independent, identical draws from a random variable Y is the same thing as one draw from each of N independent, identical random variables Yn. Now, we can talk about the sample itself as a random variable S={Y1,…,YN}."Can you please elaborate on this further? Or maybe simplify it for me as I fail to understand what you're speaking of. Thank you! – A.Asad Sep 24 at 21:01
  • @A.Asad , pick a number $N$. If you flip one coin $N$ times, that is the same as flipping $N$ identical coins, once per coin. – shadowtalker Sep 24 at 21:03
  • So what you're suggesting is that picking a random sample of size $N$ is the same as picking one number from each from $N$ identically distributed populations? This makes the sample $S=\{Y1,....,YN\}$ a random variable as it is a composition of N random variables? Thank you! – A.Asad Sep 24 at 21:09
  • @A.Asad that's correct – shadowtalker Sep 24 at 21:11
  • @A.Asad by the way, you can use the math formatting code Y_1 to produce subscripts like $Y_1$. – shadowtalker Sep 24 at 21:39

For the sake of redundancy and addition, a random value is (the Mathematical modeling of the process of having a ) a measurement or experiment whose value is not predictable/deterministic ; its value can only be understood probabilistically, meaning that it can be tested over the longer run. A standard example is that of throwing a fair coin *: one cannot tell for any throw whether it will land heads or tails, but a pattern should emerge over the long run of limiting values for the probability P(Heads)=P(Tails)=1/2.

Re your $Y$, yes, the layout is a bit confusing. Like Shadowtalker said, $Y_1,Y_2,...,Y_n$ are the implementations of the same process $Y$ , where $Y$ may represent throwing a die, flipping a coin, etc. Then $Y_1, Y_2,..,Y_n$ , if independent, are said to be IID RVs , Independent , Identically-Distributed Random Variables.

And, yes, the sampling mean is the random variable that takes sample (quantitative) values $Y_1, Y_2,....,Y_n $ and assigns to them the value $\frac {X_1 +X_2+...+X_n}{n}$. There are many other possible sample statistics: Sample variance, Sample error, etc.

An important result to note, I think, is the CLT: Central Limit Theorem which tells you that , no matter what the distribution , if $Z_1,Z_2,....,Z_m$ are independent and identically - distributed, then the sample mean will approach a normal distribution as n becomes large-enough ( $n>30 ; n>40$ for higher accuracy).

  • Assume we know it is fair to avoid a rabbit hole.
  • Well i phink modelling something probabilistically has nothing to do with the question of whether the data generating process is indeterministic. My first example would be the One you also name: a coin toss – Sebastian Sep 25 at 8:38
  • It's not the data generating process; it is the process itself that is nondeterministic. That is, AFAIK, what a Random Variable represents: A situation/experiment that can only be modeled probabilistically. Besides, isn't a coin toss modeled --probabilistically -- by a random variable that takes values in {H,T} with P(H)=P(T)=1/2 (Assuming a fair coin)? I am not sure I understand where you disagree with me: Do you disagree on my definition of an RV or on something else? – gary Sep 25 at 14:41

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.