I'm trying to understand the part of this book, page 276 which explains about sample mean:

In statistical inference, a central problem is how to use data to estimate unknown parameters of a distribution, or functions of unknown parameters. It is especially common to want to estimate the mean and variance of a distribution. If the data are i.i.d. random variables $X_1,... ,X_n$ where the mean $E(X_j)$ is unknown, then the most obvious way to estimate the mean is simply to average the $X_j$ , taking the arithmetic mean.

For example, if the observed data are $3, 1, 1, 5$, then a simple, natural way to estimate the mean of the distribution that generated the data is to use $(3+1+1+5)/4 = 2.5$. This is called the sample mean.

My question is Instead of saying "... i.i.d. random variables $X_1,... ,X_n$ where the mean $E(X_j)$ is unknown" can I write "let X be a random variable and take the $n$ outputs of $X$, where E(X) is unknown". In another words, can I see these $X_1,... ,X_n$ as only an one random variable? since the $X_1,... ,X_n$ are independent and identically distributed?

  • 1
    $\begingroup$ I think this formulation would bring more confusion than help. A random variable is at its core a function$$X:(\Omega,\mathcal P)\mapsto\mathbb R$$and to consider $n$ independent realisations the natural formulation is to consider $n$ functions. $\endgroup$ – Xi'an Sep 11 '20 at 10:33
  • $\begingroup$ @Xi'an so for each $s\in \Omega$ we take $X_i(s)=X_i$, then $X_1,...,X_n$ is in fact represented as $X_1(s),...,X_n(s)$ for the same $s$, that's correct? $\endgroup$ – user45523 Sep 11 '20 at 11:39
  • $\begingroup$ @Xi'an this isn't obvious from the text at all $\endgroup$ – user45523 Sep 11 '20 at 16:51
  • $\begingroup$ @Xi'an so suppose 1 is head and 0 is tails in a tossing game with each toss being a i.i.d. random variable. Then we had $X_i(s=\text{heard})=1$ for every $i$. Therefore my random variables would be $1,1,1,1,1,1$. $\endgroup$ – user45523 Sep 11 '20 at 17:10

Yes, rolling one die $n$ times is the same as rolling $n$ dice once, but...

"In an experimental situation, it would be very unusual to observe only the value of one random variable [...] It would be a modest experiment indeed if the only datum collected was the body weight of one person. Rather, the body weights of several people in the population might be measured. These different weights would be observations on different random variables, one for each person measured [...] Thus, we need to know how to describe and use probability models that deal with more than one random variable at a time" (Casella & Berger, Statistical Inference, Duxbury, 2002, §4.1, p. 139)

Rolling dice... ok, but measuring the body weight of one person $n$ times is not the same as measuring the body weights of $n$ people, administering a drug to one person $n$ times is not the same as amministering a drug to $n$ people.

This is why a random sample is defined as $n$ random variables, not as the reiteration of a single random variable.

  • $\begingroup$ thank you for your answer, one of the things which makes me confused is when we use a statistical software, we can type some command and it give us several outputs of a variable of a specific distribution (each time I type the command and press enter). So in your example, X could be a weight of all people and each output (each time I write the command and press enter) would be a weight of a particular person. $\endgroup$ – user45523 Sep 11 '20 at 11:26
  • $\begingroup$ This is probability, where you start with a specific distribution, and known parameters. In statistical inference you start with data, assume that they are realizations of several random variables (a random sample), try to guess the parameters of their distribution. $\endgroup$ – Sergio Sep 11 '20 at 11:41
  • $\begingroup$ Thank you very much. Now it's much clearer. The book you mentioned it's the only one which explained (some lines after your quotation) that let $s$ in the sample space then the $X_i$ are in fact $X_i=X_i(s)$ for the same $s$. That made me confused for hours. I don't know why the books on probability theory aren't explicit about that. $\endgroup$ – user45523 Sep 12 '20 at 5:50
  • $\begingroup$ Not exactly. (1) From a probabilistic point of view, a multiple random variable associates an $n$-tuple of numbers to a single point in the sample space, say $s\in \Omega=\{HHHH,HHHT,HHTT,\dots,TTTT\}$, according to a joint probability mass/density function $f(X_1,\dots,X_n;\theta)$. Here $\theta$ is the known parameter of the pmf/pdf. See Casella & Berger, Example 4.1.2: "Consider the experiment of tossing two fair dice etc." You know that the dice are fair, $\endgroup$ – Sergio Sep 12 '20 at 7:11
  • $\begingroup$ (2) From a statistical point of view, you start with $n$ observations, you assume that they are realizations of $n$ simple random variables (let's say $X_i=X(s_i)$, where $s_i\in\Omega=\{H,T\}$), assume that they are identically distributed, i.e. they have the same pmf/pdf which you do not know (you do not know $\theta$), and that they are independent, so their joint pmf/pdf is $f(X_1,\dots,X_n;\theta)=\prod_i f(X_i;\theta)$. Finally you estimate $\theta$ (e.g., whether the dice are fair or not). Probability and statistics are related but different disciplines. $\endgroup$ – Sergio Sep 12 '20 at 7:11

What you are proposing would involve a revision of probability theory, using different terminology and notation. As it stands, in probability theory a random variable denotes a single "random object" (technically it is a mapping from the sample space to some outcome space, usually a number set). We sometimes use random vectors which are vectors composed of multiple random scalars, so that is the usual way to accomodate this kind of situation. In that case we use notation like $\mathbf{X} = (X_1,...,X_n)$ to collectively denote each of the individual random variables under consideration.

What you are proposing is a different method that has some problems. Aside from being an unecessary re-invention of an existing theory that already works well, there are a few issues with the approach you are suggesting:

  • You are using the same notational object to describe $n$ different random numbers. So it then becomes quite difficult for you to express probability results where these outputs are equal to different things. For example, even simple probability statements where the outputs are equal to different things become hard to express.

  • Even assuming you can find an alternative way to express this, it would surely involve some kind of notation differentiating the $n$ outputs, and so it would be tantamount to giving them different notation to begin with. It is unclear how your method would allow you to make useful probability statements in a way that is no more complicated than in the existing theory.

  • Even assuming you can solve this issue, your method still amounts to the implicit idea that seperate "outputs" of a single random variable are IID outcomes of that random variable. So all this really does is to treat this as implicit instead of as an explicit probability statement on separate objects.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.