0
$\begingroup$

I am currently reading Deep Learning book, and I want to get better understanding of probability theory. In chapter 3.3.1 of Deep Learning book it states that:

Often we associate each random variable with a different probability mass function and the reader must infer which PMF to use based on the identity the random variable, rather than on the name of the function;P(x) is usually not the same as P (y).

And not many paragraphs latter, it saids the following:

Probability mass functions can act on many variables at the same time. Such a probability distribution over many variables is known as a joint probability distribution.P(x=x, y=y) denotes the probability that x=x and y=y simultaneously. We may also write P (x, y) for brevity

I am having hard time grasping these two paragraphs. What do they mean when they say that P(x) is USUALLY not the same as P(y). As I understood, random variable is basically a random phenomenon from a real world that we wish to model. And each random phenomenon has its own Probability Mass function. Does this mean that the first paragraph indicates that random variable y represents a different phenomenon, and in the second paragraph random variable y represents the same type of phenomenon as x and that is why we use the same probability mass function?

Thanks in advance!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Welcome to CV!

Generally in mathematics, if $f(x)$ denotes a function, $f(y)$ denotes the same function; the input variable is just a dummy variable. You could call it $x$, $y$, or anything else, but the way the function operates on an input is the same.

When they say $P(X)$ is usually not the same as $P(Y)$, they're pointing out that this is different than the above situation. It is not the case that $P$ is some function, and $X$ and $Y$ are dummy variables representing the input. For pmf's $P(X)$ and $P(Y)$ are actually different functions, even though they're both denoted by $P$.

For this reason, in most texts, the pmf's are represented by $P_Y(x)$ and $P_Y(y)$ where the subscripts $X$ and $Y$ (capital letters) denotes the random variable the pmf is for, and $x$ and $y$ (lowercase letters) are the dummy variables representing the inputs to the function.

By definition, the pmf describes the probability that a random variable takes some value. The second paragraph is just saying that if we want to know when $X$ takes some value at the same time that $Y$ takes some value, we need to use the joint pmf, $P(X, Y)$.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Hello! Thanks for the answer! However, I am still confused. I got the part where P(x) and P(y) are actually different PMF functions, and I do agree with the notation you have provided me with. But, in your last paragraph, are you saying that P(X, Y) is some third PMF function that is able to assing probabilities to both random variable X and random variable Y which represent different phenomenons? $\endgroup$
    – Stefan Radonjic
    Commented Mar 21, 2020 at 21:18
  • 1
    $\begingroup$ $P(X, Y)$ assigns probabilities to outcomes of $X$ and $Y$ occurring simultaneously. For example, say I flip a twice. Let $X$ be a 1 if the first flip is a heads, otherwise 0. Let $Y$ be a 1 if the first flip is a heads, otherwise 0. Now if we want to know the probability that we get two heads in a row ($X$ = 1 and $Y$ = 1 simultaneously), we can use $P(X, Y)$. In this case $P(X = 1, Y = 1) = \frac{1}{4}$. So yes, $X$ and $Y$ represent different phenomenons, and $P(X, Y)$ assigns probabilities to outcomes of each that occur simultaneously. $\endgroup$
    – Nick Koprowicz
    Commented Mar 21, 2020 at 22:42
  • $\begingroup$ Alright! Thanks for clarifying that! :) $\endgroup$
    – Stefan Radonjic
    Commented Mar 23, 2020 at 10:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.