52
$\begingroup$

Often, in the course of my (self-)study of statistics, I've met the terminology "$\sigma$-algebra generated by a random variable". I don't understand the definition on Wikipedia, but most importantly I don't get the intuition behind it. Why/when do we need $\sigma-$algebras generated by random variables? What is their meaning? I know the following:

  • a $\sigma$-algebra on a set $\Omega$ is a nonempty collection of subsets of $\Omega$ which contains $\Omega$, is closed under complement and under countable union.
  • we introduce $\sigma$-algebras to build probability spaces on infinite sample spaces. In particular, if $\Omega$ is uncountably infinite, we know there can exist unmeasurable subsets (sets for which we cannot define a probability). Thus, we can't just use the power set of $\Omega$ $\mathcal{P}(\Omega)$ as our set of events $\mathcal{F}$. We need a smaller set, which is still large enough so that we can define the probability of interesting events, and we can talk about convergence of a sequence of random variables.

In short, I think I have a fair intuitive understanding of $\sigma-$algebras. I would like to have a similar understanding for the $\sigma-$algebras generated by random variables: definition, why we need them, intuition, an example...

$\endgroup$
5
  • 9
    $\begingroup$ One effective (and intuitively meaningful) characterization is that this is the coarsest sigma-algebra on $\Omega$ that makes the random variable measurable. $\endgroup$
    – whuber
    Nov 7, 2017 at 15:50
  • $\begingroup$ @whuber coarsest means smallest? In other words, I have my probability space $(\Omega,\mathcal{F},P)$, I have an RV $X:\Omega\to\mathcal{R}$ (which is measurable by definition of random variable), and $\sigma$ is the smallest subset of $\mathcal{F}$ such that $X$ is still measurable. Ok, but this begs the question of what it means intuitively that $X$ is measurable :-) would it make sense to say that we can define the probability of all events of the kind $a\lt X \lt b$ and unions/intersections? $\endgroup$
    – DeltaIV
    Nov 7, 2017 at 16:23
  • 2
    $\begingroup$ Looking at a single $X$ at a time affords little intuition concerning measurability. This concept comes into its own when you study collections of random variables--stochastic processes. In turn, the simplest stochastic processes (such as finite discrete Binomial random walks) provide an interpretable setting in which the sigma-algebra generated by all variables $X_0, X_1, \ldots, X_t$ can be thought of as "the information available up to (and including) time $t$." $\endgroup$
    – whuber
    Nov 7, 2017 at 16:26
  • $\begingroup$ @whuber sorry, I don't understand :) I'd appreciate if you could point me to another answer of yours where you go more in detail, or if you would like to expand this as answer. Otherwise don't worry - maybe I don't know enough about stochastic processes to get your point. Altough..I need to hone my Dynamic Bayesian Network skills, so if this intuition helps when working on time series, I'd be quite interested. $\endgroup$
    – DeltaIV
    Nov 7, 2017 at 16:32
  • 2
    $\begingroup$ See stats.stackexchange.com/a/123754/919. Also helpful might be stats.stackexchange.com/a/164995/919 and stats.stackexchange.com/a/74339/919. $\endgroup$
    – whuber
    Nov 7, 2017 at 18:28

2 Answers 2

49
$\begingroup$

Consider a random variable $X$. We know that $X$ is nothing but a measurable function from $\left(\Omega, \mathcal{A} \right)$ into $\left(\mathbb{R}, \mathcal{B}(\mathbb{R}) \right)$, where $\mathcal{B}(\mathbb{R})$ are the Borel sets of the real line. By definition of measurability we know that we have

$$X^{-1} \left(B \right) \in \mathcal{A}, \quad \forall B \in \mathcal{B}\left(\mathbb{R}\right)$$

But in practice the preimages of the Borel sets may not be all of $\mathcal{A}$ but instead they may constitute a much coarser subset of it. To see this, let us define

$$\mathcal{\Sigma} = \left\{ S \in \mathcal{A}: S = X^{-1}(B), \ B \in \mathcal{B}(\mathbb{R}) \right\}$$

Using the properties of preimages, it is not too difficult to show that $\mathcal{\Sigma}$ is a sigma-algebra. It also follows immediately that $\mathcal{\Sigma} \subset \mathcal{A}$, hence $\mathcal{\Sigma}$ is a sub-sigma-algebra. Further, by the definitions it is easy to see that the mapping $X: \left( \Omega, \mathcal{\Sigma} \right) \to \left( \mathbb{R}, \mathcal{B} \left(\mathbb{R} \right) \right)$ is measurable. $\mathcal{\Sigma}$ is in fact the smallest sigma-algebra that makes $X$ a random variable as all other sigma-algebras of that kind would at the very least include $\mathcal{\Sigma}$. For the reason that we are dealing with preimages of the random variable $X$, we call $\mathcal{\Sigma}$ the sigma-algebra induced by the random variable $X$.

Here is an extreme example: consider a constant random variable $X$, that is, $X(\omega) \equiv \alpha$. Then $X^{-1} \left(B \right), \ B \in \mathcal{B} \left(\mathbb{R} \right)$ equals either $\Omega$ or $\varnothing$ depending on whether $\alpha \in B$. The sigma-algebra thus generated is trivial and as such, it is definitely included in $\mathcal{A}$.

Hope this helps.

$\endgroup$
8
  • 4
    $\begingroup$ $\mathcal{A}$ is the set of events, right? The one I denoted with $\mathcal{F}$ $\endgroup$
    – DeltaIV
    Nov 7, 2017 at 16:24
  • 4
    $\begingroup$ Yes, I was born with the condition of finding $\mathcal{A}$ more appealing than $\mathcal{F}$. $\endgroup$
    – JohnK
    Nov 7, 2017 at 16:26
  • 3
    $\begingroup$ excellent! Very clear. You should write a book :) $\endgroup$
    – DeltaIV
    Nov 7, 2017 at 16:26
  • $\begingroup$ Hi @JohnK, I have a question if you don't mind: in your following statement "But in practice the preimages of the Borel sets may not be all of $\mathcal{A}$." How can we prove this? The follow up statement of yours "It also follows immediately that [...] $\Sigma$ a sub-sigma-algebra." wasn't an enough of a clarification for me. I apologize and thanks in advance. $\endgroup$ May 6, 2020 at 8:51
  • 1
    $\begingroup$ @GoldmanClarck - "How can we prove this?" You can't. Paraphrasing, John was saying 'for most random variables that we come across in the wild, the preimages of the Borel sets is probably not all of $\mathcal A$'. This statement is too imprecise to prove. John does give one extreme example as evidence (the constant r.v.). $\endgroup$
    – James
    Dec 15, 2020 at 6:35
4
$\begingroup$

I will attempt to illustrate the intuition from a different perspective, less technically detailed.

Assume 4 random variables $X_1,X_2,X_3$ and $Y=f(X_1,X_2)$ for an arbitrary function $f$. Notice that $Y$ is random, but it's determined completely for fixed $X_1, X_2$, while $X_3$ is not determined for fixed $X_1, X_2$. In other words,

the randomness in $Y$ is exclusively due to $X_1$ and $X_2$.

Can we express that formally without referencing the function $f$?

This is precisely what the notion of the $\sigma$-algebra generated by a random variable captures. Informally, we could say that $\sigma(X)$ restricts the world's probabilism to just $X$, disabling any other source of randomness. In the example above, $\sigma((X_1,X_2))$ contains $\sigma(Y)$ (or $Y$ is $\sigma((X_1,X_2))$-measurable), because the randomness of $(X_1,X_2)$ contains the randomness of $Y$. The converse would be true only if $f$ is a one-to-one mapping.

$\endgroup$
2
  • 1
    $\begingroup$ Is this because of the Doob-Dynkin lemma? $\endgroup$
    – roi_saumon
    Sep 16, 2021 at 11:09
  • $\begingroup$ Yes! Nice reference. I didn't know this property had a name. $\endgroup$ Sep 19, 2021 at 9:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.