1
$\begingroup$

I'm trying to understand random variables a bit better. As I spent time thinking about, it occurred to me that the probability density of a random variable could itself be considered a random variable.

My reasoning for this is as follows:

A random variable is a function mapping each element of the sample space to the reals.

A probability density is a function mapping each element from the sample space of the random variable to the reals.

This implies that the probability density of a random variable is itself a random variable.

Is this correct?

Edit: Ok, I figured out what the thought process behind the question was. Let us that we have a random variable $X$ with an associated PDF $f_X(x)$. Clearly $f_X(x)$ is not random. However, we could define a new random variable $Y = f_X(X)$. Then $Y$ is a random variable, since it is a function of the random variable $X$. Essentially, the situation is analogous to that between the CDF $F_X(x)$ and the probability integral transform $Y = F_X(X)$

$\endgroup$
9
  • 3
    $\begingroup$ A probability density is a function mapping each element from the sample space of the random variable to the reals. Are you sure this is correct? Think what argument the probability density takes; is it an element of the sample space or rather a real number (or a real vector, for multivariate densities)? $\endgroup$ Aug 28, 2017 at 17:28
  • $\begingroup$ this might help: stats.stackexchange.com/questions/291982/… $\endgroup$
    – Taylor
    Aug 28, 2017 at 17:45
  • 1
    $\begingroup$ How is Kolmogorov complexity modern? He wrote about algorithmic complexity in 1950s or earlier @CagdasOzgenc $\endgroup$
    – Aksakal
    Aug 28, 2017 at 18:57
  • $\begingroup$ @RichardHardy It takes a real value. But now couldn't we consider the set of all real values that the random variable takes as it's sample space? In which case, the probability distribution is just a mapping from that sample space to the real line again. What am I missing? Am I misusing the term sample space? $\endgroup$ Aug 29, 2017 at 1:00
  • $\begingroup$ @InfiniteExistence, once you have the numbers, there's no more event space. If you want to build the event space on top of the values of variable, then before getting to the PDF, you need to create another random variable that maps this new event space into the numbers, and ONLY then you build PDF. PDF is always on the numbers or random variables. You can construct the PDF from the probability measure that's defined on the event space, of course. The measure is not random. $\endgroup$
    – Aksakal
    Aug 29, 2017 at 1:40

2 Answers 2

3
$\begingroup$

No.

A probability density is a function mapping each element from the sample space of the random variable to the reals.

This sounds more like the definition of the random variable itself, not its density. The random variable is the mapping of events to the numbers. For instance, you could map two heads to 1 and any other combo of tails and heads in two coin tosses to 0. This would define a random variable.

The density is function that associated these random variable values (like 0 and 1's in my example) to the probabilities. Yes, it's not a random variable. If you say that probability of 1 is 1/4, and of 0 is 3/4, then there's nothing random about this mapping.

The randomness in the density may come in the situation when you don't know the density and try to estimate it from the sample. In this case you don't know the true parameters of the density function, so you estimate them from the sample, and the parameters become random. In this sense you could say that the sample density function (through its parameters) is a random entity.

For instance, let's say you observe {0,0,1,0,1,0,0,0,1}. You could infer that $P[0]=2/3, P[1]=1/3$.

Then you get another sample: {0,1,0,1,0,0,0}, and come to a different density: $P[0]=5/7, P[1]=2/7$

$\endgroup$
3
  • $\begingroup$ @RichardHardy you have a point $\endgroup$
    – Aksakal
    Aug 28, 2017 at 18:19
  • $\begingroup$ The probability density seems to me just as "random" as the random variable. Given that an event has occurred, the value of the random variable is fixed (since it's a function). In the same way, given that the random variable has taken a value, the probability density is fixed (since it's a function). What is wrong with this reasoning? $\endgroup$ Aug 29, 2017 at 1:09
  • 1
    $\begingroup$ @InfiniteExistence, I'm not sure what's right with this reasoning. I think you have a rather peculiar understanding of what's random and what's not. Do you think that the probability of heads in a fair coin toss is random? Do you think that it's not 1/2, but something random? If you do, then I can't help you with understanding of the density. $\endgroup$
    – Aksakal
    Aug 29, 2017 at 1:33
0
$\begingroup$

Kind of (not really).

A random variable formally maps $\Omega$, your sample space, to the reals $\mathbb{R}$. The usual probability density is defined from the reals to the reals, so from this point of view, it's not a random variable. For example, you can define $\Omega$ to be the sigma-algebra of coin flips $H,T$ and then $X(H)=1$ and $X(T)=0$ would define a Bernoulli random variable. Its density (pmf in this case on $\mathbb{Z}$) $f(x)=\frac{1}{2}\delta(x)+\frac{1}{2}\delta(x-1)$ wouldn't be a function from $\Omega\rightarrow \mathbb{R}$, but rather $\mathbb{Z}\rightarrow\mathbb{R}$.

On the other hand, there are high-brow theorems that state that instead of using $\Omega$, you can think of a random variable defined by its push-forward measure, specifically through $P(X\leq x)$. For example when you want to define a standard normal random variable, you can do away with the need for $\Omega$ by instead defining $P(X\leq x)=\int_{-\infty}^x\frac{1}{\sqrt{2\pi}}\exp(-t^2/2)dt$ (and correspondingly it's density).

This procedure actually defines a probability measure on $\mathbb{R}$ through $P_X((-\infty,x)):=P(X\leq x)$. Thus we can assign $\Omega=\mathbb{R}$ and then the normal random variable becomes $X:\Omega\rightarrow \mathbb{R}$, where we equip $\Omega=\mathbb{R}$ with $P_X$. Now when you have the density $f_X(x)$, you can turn it into a random variable on $\Omega=\mathbb{R}$, with measure $P_X$.

$\endgroup$
5
  • $\begingroup$ Not really relevant to your point, but it's kind of weird to use a discrete example here, which explicitly doesn't have a density and where the $f$ you wrote is not actually a function from $\mathbb R \to \mathbb R$ like you said it is. (And even less relevantly, three of the four times you said "it's" should be "its." :p) $\endgroup$
    – Danica
    Aug 28, 2017 at 17:41
  • $\begingroup$ @Cagdas It's hard to see how that comment could be relevant (or even correct): could you explain how such an "encoding" would work and what it might have to do with the question? $\endgroup$
    – whuber
    Aug 28, 2017 at 20:40
  • $\begingroup$ @Dougal: Points taken. It's hard to think of an "easy" example of a random variable defined on an abstract sigma-algebra space, so I opted for one with a mass function instead of a density. $\endgroup$
    – Alex R.
    Aug 28, 2017 at 20:56
  • $\begingroup$ @CagdasOzgenc, it seems you are deleting your early comments but still proceeding to argue further. I think this makes understanding your points harder. (I would be genuinely interested in early comments as much as the latter ones to be able to capture the line of thought.) $\endgroup$ Aug 29, 2017 at 6:43
  • 1
    $\begingroup$ Then it would seem to me that such discussion should be held in a chat room. Otherwise it still clutters the comment space and only you and the intended recipient understands what is going on. This is just my opinion, of course. Also, I believe most of the users are fine with diverse ideas as long as they are presented in an objective and neutral tone. I know myself that some guys here are very adamant about defending some preferred ideas as the best ones, but as long as the discussion is strictly objective, users with other ideas would not be suspended for holding a different point of view. $\endgroup$ Aug 29, 2017 at 8:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.