11
$\begingroup$

A frequent simplification in modeling and simulation is to replace a random variable by its mean value.

When would this simplification lead to the wrong conclusion?

$\endgroup$
4
  • 2
    $\begingroup$ Does "Var" stand for variable or variance or Value At Risk? $\endgroup$
    – Henry
    Oct 24, 2017 at 22:24
  • 3
    $\begingroup$ It'd be fun to start a service that pays for its members' Netflix subscription. We'd charge only $\left|x\right|~\frac{\mathrm{USD}}{\mathrm{month}}$, where $x$ is randomly selected in the domain $\left[-100,100\right]$, so, ya know, free Netflix! Later, we'll offer some customers the option to instead pay $x^2~\frac{\mathrm{USD}}{\mathrm{month}}$. $\endgroup$
    – Nat
    Oct 25, 2017 at 7:42
  • 3
    $\begingroup$ Well in a very simple case if we take it to the the extreme we could lose pretty much all the information we care about. Consider a regression of Y on X where we replaced both Y and X with their mean. Any information about the slope is now lost. $\endgroup$
    – Dason
    Oct 25, 2017 at 12:48
  • 1
    $\begingroup$ Are you asking about replacing missing values, or are you asking about a replacing a random variable in a specific context (e.g. making predictions base on a random-effects model)? $\endgroup$
    – IWS
    Oct 27, 2017 at 9:29

3 Answers 3

20
$\begingroup$

If you replace a missing value by some point estimate, you disregard all its variability. Thus, you will not propagate all the original variability to your model. Your parameter estimates will appear to have too low s. If you do inference, your p values will be biased low. Your s will be too narrow. If you do prediction, your s will be too narrow.

Overall: you will be too sure of your conclusions.

$\endgroup$
7
  • 2
    $\begingroup$ Good answer! Think about this way: A random variable has a distribution. It can be skweded to the left, to the right. I can be bi-modal etc. By reducing the variable to it's mean value you are removing all that extra information (uncertainty) and replacing a distribution (intervals) by a single point estimate. $\endgroup$ Oct 25, 2017 at 10:12
  • 1
    $\begingroup$ If you replace a missing value by some point estimate, you're also assuming the data is missing at random. The mean value of the random variable might not equal the mean value of the data when it's missing. $\endgroup$
    – Neil G
    Oct 25, 2017 at 16:03
  • $\begingroup$ @NeilG sorry to nitpick, but replacing a missing value by its mean does not directly mean assuming the data to be missing at random. Especially since the - somewhat confusing - terminology around missing data considers 'missing at random' to be data that is missing at random conditional on other, but known data (en.wikipedia.org/wiki/Missing_data). IMO, the way data is replaced does not imply anything about the reasoning behind it. That reasoning should be made explicit and lead up to the appropriate way of handling the missing data. That said, I fully agree with Stephan's answer. $\endgroup$
    – IWS
    Oct 27, 2017 at 9:28
  • $\begingroup$ @IWS It's fine for the missingness indicators to be conditional on the observed data. Missing at random means that the missingness indicators depend on the unobserved data. If you replace the variable with its mean value conditional on it being observed, that might not be the same as its unconditional mean value — unless the data are missing at random. $\endgroup$
    – Neil G
    Oct 27, 2017 at 9:46
  • $\begingroup$ @NeilG Don't you mean 'missing completely at random', when you write 'missing at random' in the final sentence of your last comment? If so, we do are in agreement, but I was just nitpicking about terminology. (see the wiki page I've put in my comment above, I've always been taught, read and used that terminology) $\endgroup$
    – IWS
    Oct 27, 2017 at 9:51
14
$\begingroup$

In addition to Stephan's points:

  • In almost any application where you're interested in nonlinear functions of the random variable, substituting the mean will generally introduce bias and possibly contradictory results. The average velocity and average mass of a particle will generally not be consistent with average kinetic energy, because energy scales with V^2.
  • The mean value may not even be a possible outcome for the random variable. If my possible outcomes are 0 "patient dies" and 1 "patient lives", it's probably not helpful to have a model that describes the patient as 0.1 "mostly dead but slightly alive".
$\endgroup$
2
0
$\begingroup$

A real life example (related to the two answers you got), in the financial markets. The price of an option is based in the probability that the price of an asset goes above (or below) a given level.

For example, the price of an option for buying an asset at a price 100 when the expected value of the asset is 80. If you substitute the random variable (the asset price) by its mean, you would get a price of zero (as you would never by at 100 an asset that costs 80). When you take into account the stochasticity of the asset (and that's the right way of doing it) you get a positive price, as there is some probability that the asset price goes above 100.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.