8
$\begingroup$

I am not sure what the right keywords would be for this but I would like to know if it is possible to apply functions to random variables. I think it may make sense in terms of expected value but I would appreciate any information on a more formal or rigorous approach to this concept.

For example if there was a random variable X and a function f(x)=2*x and E[X] = 2 would E[f(X)] = 4 ?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ According to an elementary, rigorous account of random variables, $X$ is a way of writing numbers on tickets in a box. A function $f$ is a rule to erase the value $X(\omega)$ and replace it by $f(X(\omega))$ on each ticket $\omega$. That should make it plain that (a) functions can be applied to random variables, (b) the sense in which they are applied, and (c) there actually is a technical condition they must satisfy: namely, for every $x$, $\{\omega|f(X(\omega))\le x\}$ must be measurable. $\endgroup$
    – whuber ♦
    Commented Aug 25, 2013 at 17:16

2 Answers 2

Reset to default
9
$\begingroup$

Yes you can definitely have functions of random variables. And yes, in the case you present above

$$E[f(x)]=4$$ however, this is only because the expectation works as a linear operator. Had the function been more complex you could not apply what you did as easily. For sake of argument consider the random variable $X$ and the function of the random variable $f(X)=X^2$. If $E[X]=2$ then we cannot conclude that $$E[f(x)]=E[X^2]=E[X]^2$$ as the logic from above would follow. Instead we would need to calculate new distribution of our new random variable $f(X)$ (yes a function of a random variable is a random variable itself). Now calculating the distribution of our new random variable is not always trivial though and there are more than one way to do. The straight forward brute force way is the following:

$$F(f(X))=\text{Pr}(f(X)\leq x)=\text{Pr}(X\leq f^{-1}(x))$$ and to continue on in that manner, however, this assumes that we have a function $f$ that is invertible. A convenient method for when the function of the random variable is a strictly monotone function over the support of $X$ we have the following: Let $Y=f(x)$

$$p_{Y}(y)=p_X(f^{-1}(y))\bigg|\frac{df^{-1}(y)}{dy}\bigg|$$

If you want to read more about this I would say google functions of random variables, transformations of random variables or look here: http://en.wikipedia.org/wiki/Random_variable

Also, any introductory textbook on probability theory will cover all of this in great detail.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you so much, this is exactly what I was looking for! $\endgroup$
    – Twiltie
    Commented Aug 21, 2013 at 21:09
  • $\begingroup$ @BabakP It's not correct that "we would need to calculate new distribution of our new random variable $f(X)$". You can use the theorem $E(f(X)) = \int_{-\infty}^\infty f(x) p_X(x) \, dx$ (en.wikipedia.org/wiki/Law_of_the_unconscious_statistician) $\endgroup$
    – Luis Mendo
    Commented Aug 21, 2013 at 23:37
  • 1
    $\begingroup$ @Luis, yes I wholeheartedly agree with you that there is more than one approach, however, I was trying to make the point that $X$ and $X^2$ have two different distributions. I guess at the end of the day I should have phrased it as "we could calculate new distribution of our new random variable f(X)." $\endgroup$
    – user25658
    Commented Aug 22, 2013 at 0:35
  • $\begingroup$ Also, @Luis, your comment misses the point that outside of expectations (and maybe some other operators) you need to calculate the distribution of the new random variable (that is a function of the old random variable) to proceed. $\endgroup$
    – user25658
    Commented Aug 22, 2013 at 0:37
  • 1
    $\begingroup$ Also, try applying the law of the unconscious statistician when you do not have a joint pdf and but only marginals pdfs $p_X(x),p_Y(y),$ and $p_Z(z)$, and assuming the marginals are not independent, and you want the expectation of $f(X,Y,Z)=(X+Y)/Z^3$. You are going to have to find a suitable transformation along the lines of what I was proposing. $\endgroup$
    – user25658
    Commented Aug 22, 2013 at 3:37
4
$\begingroup$

BabakP's answer is a good one. Read it, but I'd like to add a few points.

The computation of the distribution of a transformation of a random variable (or of several variables) is often called statistical tolerancing. A problem is, for most such transformations, there is no simple distribution that works. Thus suppose X is a Gaussian random variable, then F(X), where F is a linear function, also has a Gaussian distribution, but with an adjusted mean and variance, based on the transformation L, and the mean and variance of X.

However, things get nasty almost always when F(X) is at all nonlinear. Ok, if F is the exponential function, then F(X) for normal X has a lognormal distribution. But most nonlinear functions will not give you some well known distribution. So what can you do?

One simple solution is to compute the mean and variance of the transformation. Given a mean and variance, one might choose to assume a normal distribution with that mean and variance. One might even do more, computing the first four moments of F(X). There are several families of distributions (Pearson & Johnson families) that allow you to find a distribution that matches the moments you have just found.

So, the question is, how might one compute those moments? The simple answer is if you knew the derivative of F, then approximating F by a truncated (first order) Taylor series around the mean of X can allow you to find approximations for those moments. Essentially, if F is well approximated by a linear function over the support of X, then those moments will be good estimates. (For a Gaussian distribution, the support might be considered to be something like +/-6 sigma.)

Others have used second order Taylor series approximations. Here too we can compute approximate moments of the transformation.

And finally, I might mention Taguchi methods, as well as modified Taguchi methods. They too allow you to find approximations of the moments. A nice thing about the modified Taguchi methods is they are based on Gaussian integrations, and allow you to use higher order approximations quite easily, without any need to compute the derivatives of your transformation F.

Another nice feature of the modified Taguchi methods is they easily allow you to formulate a method that works on uniform random variables X, or gamma random variables, etc. In fact, there are schemes that will allow you to solve for the moments of a wide variety of random variables.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for your answer! It appears I have quite a bit of reading to do if I really want to grasp what happens if I apply anything other than simple linear functions. $\endgroup$
    – Twiltie
    Commented Aug 22, 2013 at 14:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.