Are data considered to be "events" or "random variables" in machine learning?

Question

I was sitting at a lecture on Naive Bayes, and the speaker, on a slide, said:

Given a feature $x = \begin{bmatrix} x_1, \ldots, x_n \end{bmatrix}^T$, the probability of the feature belong to class $c$ is given by,

$P(c|x) = P(x,c)/P(x) = P(x|c)P(c)/P(x)$

I was a bit surprised by this notation.

From Wikipedia, the Bayes rule only apply for either "events" or "random variables".

https://en.wikipedia.org/wiki/Bayes%27_theorem#Derivation

Is the feature (or data) $x$ considered to be event or random variable in this context?

Answer 1

Typically in machine learning (or statistics), we don't think of features/covariates as random variables. Or at least, we often don't care about the random nature of our features. We just want to model $P(y | X)$, where $y$ is the outcome of interest, and $X$ is our set of covariates. We care about the random nature of $y$ given $X$, but we assume $X$ is known so we don't bother with modeling the random nature of it.

Naive Bayes is a special case, though. Under Naive Bayes, we think of things in terms of a hierarchical process. At the very top is the outcome $y$. This is simply a binomial random variable (assuming binary outcomes). Then, our features follow a distribution conditional on the value of $y$. If we know the $P(y = 1)$ and $P(X | y = 0)$ and $P(X | y = 1)$, we can use Bayes Theorem to compute $P(y = 1 | X)$. The key point here is that using this approach, we need a model for $P(X | y)$ to make inference about $P(y | X)$.

So for this particular model, you do need to think about the distribution of $X$, in contrast with most other models that allow you to directly compute $P(y |X)$ and ignore the random aspect of $X$.

Answer 2

You mean using uppercase $P$? But then, if those were events they'd also use uppercase letters for the events. While people usually use uppercase $P$ for probabilities, not probability mass functions, same with uppercase letters for events, but there's nothing strict about it, as notation in math is flexible and can be ambiguous unless considering the context. Moreover, Bayes theorem is the same no matter if we are talking about probabilities, probability mass functions, or probability density functions.