0
$\begingroup$

I apologize in advance if this sounds like a really bad question.

Probability and statistics is extensively used in machine learning, and random variable is extensively used in probability and statistics.

However, from my observation, the concept of the random variable barely shows up when people are actually doing machine learning or conducting machine learning related research.

Examples,

  1. People in machine learning uses the notation $p(x)$ to denote a probability distribution and $p(x|y)$ to denote a conditional distribution. It makes zero references to the underlying random variable. Clearly, the usage of the random variable is not necessary as this notation is employed in over 90% of machine learning research.

  2. People in reinforcement learning are very happy to use the notation to denote the expected return

$$\mathbb{E}[R_t|s_t,a_t]$$

Here, it is obvious $s_t, a_t$ are the current state and current action and the outcomes of the respective random variable rather the random variables themselves. But once again, we have shortcircuited the usage of the random variable to describe our model. The notation makes even less reference to the random variable the deeper you go into this field.

  1. The random variable is a function $\Omega \to \mathcal{X}$, where $\Omega$ is the set of possible outcomes of an experiment, and $\mathcal{X}$ is the number associated to them by a random variable. But suppose our task at hand is image generation, i.e., GAN: we are trying to generate all possible human faces. What could even be a description of the set of all possible human faces? This set is way too large to be written down, enumerated or even be imagined (much larger than the number of atoms in the universe). Hence this function is not useful at all. What is useful is the image is actually generated, which is a vector $x \in \mathcal{X}$.

Again, it seems to me that the concept of a random variable here is useless (and is useless whenever the underlying sample space is too abstract or too large, as is most of the times in machine learning related applications).

See related question: Sample Space of Machine Learning Classification "Experiment"


Do we even need this concept at all?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.