Statistics, Expected values and Philosophy

First a note: I am not a statistician. I studied maths at university (but opted out of every single stats class), and now find myself in a job where I'm doing stats.

My question is a little bit philosophical, and I'm sure that I have my brain in a twist by thinking too hard about the wrong things, but I'm struggling with the concept of a random variable.

Suppose a typist can type on average $\mu$ words per minute. If we take a sample of his (or her) typing we could approximate a normal distribution with mean $\mu$. I think that is pretty standard.

So we can say we expect the typist to type $\mu$ words per minute. OK, what if we replace the typist with another person? Are we to expect the second person to type at $\mu$ words per minute? Of course not, because we only have a sample for the first person and not the second, right? The second typist's mean could be completely different. That makes sense, but then what if I put the original typist back in and tie one hand behind his back. Obviously the expected value is different, because the typist clearly can't type as quickly. But then what conditions can we expect the typist's average to be $\mu$? It seems any perturbation in the typist's state changes the probability. But we never really defined the typist's state to begin with, the expected value was calculated from a sample. And obviously every time the typist types he or she is in a slightly different state, i.e. maybe has a headache, maybe fingers ache, maybe daydreaming, but we still count those in our sample. So it seems to me, that technically, if we can accept some small changes in state, then why can't we accept big changes in state? Or where is the cutoff? i.e. If I replace the typist, I have just as much right to say that the expected words per minute is still $\mu$.

I would like someone to explain why the above is wrong. Maybe it's an obvious answer.

• The symbol $\mu$ is widely used in statistics to refer to a mean, but the symbol $x$ does not have that connotation. I've simplified your notation to refer to $\mu$ alone by an edit; I don't think you were trying to make a distinction with your notation. On a different note, "expectation" in statistics does have the specific sense of what you expect on average and mentioning an expectation is entirely consistent with expecting variations around it. – Nick Cox Jun 14 '13 at 15:02

I think the answer to your question is best answered by focusing on what it is, precisely, that you want to know.

You're correct that the typist's average words-per-minute is only a meaningful inference on that particular typist. But measuring the typist's performance at many intervals will, hopefully, randomly mix in those other states and therefore average over them. The mean value can be seen as the weighted average of all of these different states. If you wanted, you could construct a model to incorporate other explanatory information which might improve or degrade the typist's abilities -- headaches, poor sleep, etc. But this is probably only necessary if you need to measure the importance or effect size of some circumstances on the typist.

If you're interested in inferences across the universe of typists, you would clearly have to collect information on more than that one typist. Using a large random sample the typing pool, you could describe the rate of the typical typist, their dispersion, etc. Even though their rate is likely a complicated function of biological, physical and environmental factors, and therefore deterministic rather than stochastic, if we are not interested in those features, and we believe that our random sample is representative of these characteristics, our inferences will effectively "average over" these characteristics.

If our research is instead concerned with the covariates of typing, then it would be essential to collect information on the typists' condition -- maybe their headache status, whether they have arthritis, their age, etc.

I hope that this provides the context to understand the logic underlying statistics and addressed the questions you've raised. Let me know if you would like further elaboration, and welcome to Cross Validated.

DJE gave a great answer, but I would just add some clarification about which states we can and can't accept.

When you measure the typist's speed, you are really measuring the marginal expected value. For simplicity, let's consider the typist's speed to be in one of two states (fast =1, slow=0) and say we are trying to estimate the probability that this is a fast typist.

So, when you measure the speed over several time points and average these, you are estimating Pr(fast). But by selecting time points at random, you are estimating this via the contributions to the marginal probability by the probability that the typist is fast given those states that occur for the typist: Pr(fast) = Pr(fast|headache)Pr(headache) + Pr(fast|tired)Pr(tired) + Pr(fast|hungry)Pr(hungry) + Pr(fast|deadline)Pr(deadline)+... etc.

If you select your time points randomly, then the states that occur in your dataset should be representative of the distribution of states that occur in the population of this typist's time. Very rare states may be missed, but unless they have a large effect on typing speed this will not cause much bias. If your typist ever spends time with her hand tied behind her back while typing, then Pr(hand tied) >0 and it will be represented in your data with probability = Pr(hand tied). The reason we don't accept 'having one hand tied behind the typist's back' as a state is because this is not a normal variation in the typist's state - i.e. we believe that Pr(hand tied) = 0. Similarly, if Pr(typist A = typist B) = 0, then we don't accept switching typists as a state to which our marginal probability applies.

This question hinges on the probabilistic concept of exchangeability. Consider an experiment that generates a series of outcomes $$X_1, X_2, X_3,...$$ and suppose that we have "stable conditions", by which we mean that the conditions of the experiment are sufficiently similar in essentials from trial to trial that we are confident that the order of the outcomes is not informative. This "stability" in the experiment is captured by the notion of exchangeability of the sequence of outcomes. Indeed, we can reasonably define "stable conditions" in an experiment as being conditions that we believe lead to exchangeability of the outcomes.

Parametric estimation in an experiment can be understood operationally as follows. If we have an experiment that can generate a (hypothetically infinite) sequence of outcomes $$X_1, X_2, X_3,...$$ then we can define the parametric mean:

$$\mu \equiv \lim_{n \rightarrow \infty} \frac{1}{n} \sum_{i=1}^n X_i.$$

Under the condition of exchangeability, it can be shown that $$\mathbb{E}(X_i) = \mu$$ for every outcome in the sequence. Philosophically, this is operational result for an infinite sequence is the justification for considering this "parameter" at all. That is, the "parameter" is only a thing if we assume that there is some infinite sequence of values from which it is operationally defined. From the common mean that arises under exchangeability, the sample mean $$\bar{X}_n = \sum_{i=1}^n X_i / n$$ is a reasonable estimator of $$\mu$$ that is unbiased and strongly consistent.

Now, if we change the conditions of the experiment (e.g., by changing the typist, or changing the conditions under which the original typist operates) then we can no longer say that the combined outcomes of those two experiments are exchangeable outcomes. Hence, we can no longer appeal to the common mean for an exchangeable sequence of random variables, and we no longer have any reason to believe that the sample mean from one experiment will act as a reasonable estimator of the parametric mean of another. In this case we now effectively have two separate sequences of outcomes, which might be individually exchangeable (if each has stable conditions in its own right), but which are not exchangeable when fused together.

Understanding the philosophical basis for this issue requires a deep familiarity with the IID model derived from an exchangeable sequence. In this regard, I strongly recommend reading up on exchangeability, and operational models of parameters (see e.g., Bernardo 1996).

If you consider x to be constant, well the expected value will be x. If you want to look at the evolution of x you cannot make the hypothesis that x is constant.

My approach would be different. Suppose now that your x is evolving over time, you have a time series.

To guess the expected value you can build a model. This model should include:

A trend: x will grow as the typist learns how to type, will decrease as the typist gets older.

Periodical effects: Annually as motivation come and go with the weather, Monthly as motivation come and come with the pay, daily as the typist can feel fatigue.

And some high variation: sickness if you look at days, loss of attention if you look at hour, typo if you look at minutes.

As you can see, taking a mean as an estimator of the future is not enough. You have to build a good estimator. This could be complex as I probably forget some effects (fear of the blank page, a publication to finish ... ) and as it will need lots of x measures.

So we can come back to your solution.

Suppose the trend is slow. Choose a period wich will keep the periodical effects you need and avoid the others. Build a model for other effects. A simple one can just take the probability of being affected and the loss of speed.

You can take the mean on the period mentioned above, and adjust for other effects to have a good estimation of the expected value.