Consider the situation where one can distinguish between two different event $A$ and $B$, where the first event $A$ is characterized by a random variable $x_{A}\sim P_{A}\left(x_{A} \right)$ and the second event $B$ is characterized by a second random variable $x_{B}\sim P_{B}\left(x_{B}|x_{A} \right)$, respectively ($x_{A}, x_{B}\in \mathbb{R}$). Note that while the random variable $x_{A}$ of the first event $A$ is distributed independently of the outcome of event $B$, the outcome of event $B$ is generally conditioned on the outcome of event $A$. Therefore, one can also write $(x_{A}, x_{B})\sim P_{AB}\left(x_{A},x_{B} \right)$, where by the rule of conditional probabilities \begin{equation} P_{AB}\left(x_{A},x_{B} \right) = P_{A}\left(x_{A} \right) P_{B}\left(x_{B}|x_{A} \right). \end{equation}

Suppose we are then interested in the average of some function $f: (x_{A}, x_{B}) \rightarrow \mathbb{R}$. Meaning, we want to compute \begin{equation} \langle f \rangle = \int\int P_{A}(x_{A}) P_{B}(x_{B}|x_{A}) f(x_{A}, x_{B})dx_{A}dx_{B}. \end{equation}

When modelling these two events we ignore their dependence and describe them as two independent events governed by probability densities $P_{A}(x_{A})$ and $P_{B}^{\rm eff}(x_{B}) = \int P_{A}(x_{A})P_{B}(x_{B}|x_{A})dx_{A}$. Then \begin{equation} \tilde{\langle f \rangle} = \int\int P_{A}(x_{A}) P_{B}^{\rm eff}(x_{B}) f(x_{A}, x_{B}) dx_{A}dx_{B} = \int P_{B}^{\rm eff}(x_{B}) f^{\rm eff}(x_{B}) dx_{B}, \end{equation} with $f^{\rm eff}(x_{B}) = \int P_{A}(x_{A}) f(x_{A}, x_{B}) dx_{A}$.

Making this model assumption, what is my error $E=\langle f \rangle - \tilde{\langle f \rangle}$? Or in other words, can one detect a flawed model which ignores the dependence of two events just from computing an average quantity $\leftrightarrow$ is $E \neq 0$? There are certainly common ways to figure out whether two events are independent or not; is a computation of $E$ one of them? What's the intuition behind this fact? Intuitively it should certainly be harder to detect the dependence of events if one does not have full access to the underlying probability densities, i.e., access to the probability of individual sequences of events ...

  • $\begingroup$ @Xi'an By an event I simply mean: You draw one sample from the underlying probability distribution which governs the event. Event $A$ and $B$ can, for example, be thought of as sequential dice tosses (except that the outcome may, in general, be distributed continuously and characterized by a probability density and not a probability distribution). $\endgroup$ Nov 2, 2020 at 17:19
  • $\begingroup$ @Xi'an Event $A$ and $B$ are guaranteed to occur. For example, we are guaranteed to observe two dice tosses - we are simply set out to model this sequence of events. $\endgroup$ Nov 2, 2020 at 17:22
  • $\begingroup$ Thus they should not be called "events" but are simply indices for the two random variables. $\endgroup$
    – Xi'an
    Nov 2, 2020 at 17:24
  • $\begingroup$ @Xi'an I am sorry if I did not use the "common" language of statistics and caused confusion... $\endgroup$ Nov 2, 2020 at 17:26

1 Answer 1


If \begin{equation} \langle f \rangle = \int\int f(x_{A}, x_{B})\text{d}P_{A}(x_{A})\,\text{d}P_{B}(x_{B}|x_{A}) \end{equation} then, assuming Fubini's theorem holds (for instance assuming $f(\cdot,\cdot)\ge 0$) \begin{align} \langle f \rangle &= \int\left\{\int f(x_{A}, x_{B})\text{d}P_{B}(x_{B}|x_{A})\right\}\text{d}P_{A}(x_{A})\\ &=\int\left\{\int f(x_{A}, x_{B})\frac{\text{d}P_B(x_B|x_A)}{\text{d}P_B^{\rm eff}(x_B)}\text{d}P_{A}(x_{A})\right\}\text{d}P_{B}(x_{B})\\ &=\int\underbrace{\left\{\int f(x_{A}, x_{B})\text{d}P_{A}(x_{A}|x_{B})\right\}}_{\mathbb E[f(X_A,x_B)|x_B]}\text{d}P_{B}^{\rm eff}(x_{B})\\ \end{align} which clearly differs from $$\int\left\{\int f(x_{A}, x_{B}) \text{d}P_{A}(x_{A})\right\}\text{d}P_{B}^{\rm eff}(x_{B})$$ How much they differ will depend on the choice of the function $f$ and the dependence between $X_A$ and $X_B$. Considering $$\sup_{f;\ \langle f\rangle=1}\left| \int\int f(x_{A}, x_{B}) \text{d}P_{A}(x_{A})\text{d}P_{B}^{\rm eff}(x_{B}) - 1\right|$$ gives a measure of dependence between $X_A$ and $X_B$ connected with notions like $\alpha$-mixing, $\beta$-mixing and other measures of dependence.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.