2
$\begingroup$

This formula is taken from the book Mathematics for Machine Learning (p. 177, Formula 6.8). I don't know how to transform from the left to the right side. I searched for probabilities rules, I looked through the symbols and I totally don't understand.

$$ P_X(S) = P(X \epsilon S) = P(X^{-1}(S)) = P(\{\omega \space \epsilon \space \Omega : X(\omega) \varepsilon S \}) $$

$\endgroup$
4
  • 2
    $\begingroup$ These are all notational manipulations originating in the basic language of mathematics (set theory, mostly). The first can be taken as a definition of the "$P_X$" notation, while the others explain what "$X\in S$" might mean and then define$X^{-1}(S).$ None of them has anything to do with rules of probability. If your book doesn't define this notation, then a good reference would be an elementary book on set theory, such as that by Paul Halmos. You don't need to learn set theory: the notation will be defined within the first few pages. $\endgroup$
    – whuber
    Commented Dec 30, 2021 at 21:26
  • 2
    $\begingroup$ A quick skim of that book indicates nobody is going to learn mathematics from it--too much is assumed to be a "review" and a great deal goes either undefined or badly defined. If you must use it, then consider supplementing it with introductory textbooks on relevant subjects. $\endgroup$
    – whuber
    Commented Dec 30, 2021 at 21:40
  • 1
    $\begingroup$ @whuber: For your information, I notice that the book has an issues page where you can post your comments, especially if you believe the book is failing in its goal to "motivat[e] people to learn mathematical concepts" for machine learning. $\endgroup$
    – Peter O.
    Commented Dec 31, 2021 at 4:25
  • $\begingroup$ Thank you PeterO. and whuber for your guidance! $\endgroup$ Commented Dec 31, 2021 at 22:11

1 Answer 1

1
$\begingroup$

In order to understand this equation you will need to go back to the definition of a probability measure in measure theory. The book you are working from is not a good introductory book for that purpose, so I recommend you start by reading an introductory text on probability and measure. In any case, I will give you an explanation of how this equation comes about.$^\dagger$ In this explanation I will refer to "measureable" sets, for purposes of rigor, but you can ignore that part until you know more about measure theory.


Explaining the equation: The symbol $P$ in this equation is referring to a probability measure on the abstract sample space $\Omega$. We call the elements $\omega \in \Omega$ "outcomes" in the sample space and we refer to (measureable) sets of these outcomes as "events". The probability measure takes an event and tells us its probability.

Now, in most applications of probability theory, we want to work with random numbers, and to do this, we label each of the outcomes in the sample space by a number. In the present case, $X(\omega)$ is the number ascribed to the outcome $\omega$ in the sample abstract space. Suppose you want to find the probability that the number $X$ falls within some particular set of numbers. To do this, we define a new probability operator $P_X$ which takes in a set of numbers instead of a set of outcomes in the abstract sample space. This is really a definition of the new probability measure:

$$P_X(\mathcal{S}) \equiv P(X \in \mathcal{S}) \quad \quad \quad \text{for all measureable } \mathcal{S}.$$

Since $X: \Omega \rightarrow \mathbb{R}$ is a function mapping the outcomes in the abstract sample space to the real numbers, the pre-image $X^{-1}$ for this function is:

$$X^{-1}(\mathcal{S}) \equiv \{ \omega \in \Omega | X(\omega) \in \mathcal{S} \}.$$

Using standard properties of the pre-image, we have the event equivalence:

$$X(\omega) \in \mathcal{S} \quad \quad \iff \quad \quad \omega \in X^{-1}(\mathcal{S}).$$

Putting all these things together means that for any measureable set $\mathcal{S}$ we have:

$$P_X(\mathcal{S}) = P(X \in \mathcal{S}) = P(X^{-1}(\mathcal{S})) = P(\{ \omega \in \Omega | X(\omega) \in \mathcal{S} \}).$$

This final equation essentially just says that the probability that $X$ is in the set $\mathcal{S}$ (using the probability measure $P_X$) is the same as the probability that we have an $\omega$ with $X(\omega) \in \mathcal{S}$ (using the probability measure $P$). The purpose of the equation is to show how the measure $P_X$ relates back to the foundational probability measure $P$.


$^\dagger$ I don't really like the kind of notation used in the book, so I'm going to use some slightly different symbols/fonts for this equation; it is the same thing in slightly different notation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.