Why data points are thought as random variable

Question

I'am currently following a basic statistics course and a machine learning course. I try to understand what is covariance. In general, books define covariance as follows: Covariance is a measure of how much two random variables vary together. Okay, I got the definition, I understood the formula of covariance etc. However, when I search on the web to find an example where covariance calculated, I found a video on youtube. Starting from 43 seconds in that video, she tries to find the covariance of set of 4 data points. This is the part I confused. How the random variables are related with data points ? Is it because these data points are sampled from a distribution we don't know and if we talk about a distribution there have to be a random variable ? If this is the case, isn't it a continuous random variable ? Is it possible to calculate the covariance of 2 continuous random variables in the same way as it is done for discrete random variables ?

Answer 1

Let $X$ and $Y$ be two real random variables defined on a probability space $(\Omega,\mathcal{F},\mathbb{P})$, where $\Omega$ is a set of outcomes $\omega$. They then also can be seen as functions $X\colon \Omega \to \mathbb{R}$ and $Y\colon \Omega \to \mathbb{R}$. That is, e.g., $X(\omega)\in \mathbb{R}$ for some $\omega\in\Omega$ is a number, called a realization of $X$, and that is precisely what we observe in practice. (That is, data points are realizations of random variables, contrary to your question title.)

As an example, let $X$ and $Y$ be a person's height and weight, and let $X_1, Y_1, \dots, X_n, Y_n$ be independent and identically distributed copies of $X$ and $Y$, so that we are dealing with $n$ people. We may be interested in knowing $$ \operatorname{\mathbb{C}ov}[X,Y]=\mathbb{E}[XY]-\mathbb{E}[X]\mathbb{E}[Y]. $$ While $X$ and $Y$ indeed are random variables (or functions) that we do not expect to fully ever now, in practice we may observe realizations $X_1(\omega), Y_1(\omega), \dots, X_n(\omega), Y_n(\omega)$. Now it is natural to expect that, if $n$ is large enough, we may get a pretty good idea about $X$ and $Y$.

For instance, if you knew that those $n$ people in theory are equally likely to be of any height $h$ (i.e., they heights are identically distributed), then after knowing $X_1(\omega), \dots, X_n(\omega)$ you probably would be quite confident that $\mathbb{E}[X]$ and $n^{-1}\sum_{i=1}^n X_i(\omega)$ are really close.

The same happens with covariance. When $n$ is large enough and certain regularity conditions hold, using various law of large numbers we get that those empirical estimates $\widehat{\operatorname{\mathbb{C}ov}}_n[X,Y]$ that we compute in practice do indeed come close to the true $\operatorname{\mathbb{C}ov}[X,Y]$.