# Express expectation value of a joint distribution over a discrete and continuous random variable

Let $$Y$$ be a discrete random variable and let $$X$$ be an (absolutely) continuous random variable and $$f(X, Y)$$ a function of these two random variables. Let $$P(X, Y)$$ be the joint probability measure. I am now wondering how to properly write the joint expectation value $$\mathbb{E}[f(X, Y)]$$? I would write something like: $$\mathbb{E}[f(X, Y)] = \sum_{y} \int f(x, y) dP(X=x, Y=y),$$ but I think this cannot be quite right because I shouldnt integrate over y. So I would like to know how to write it down, both in terms of a Lebesgue integral, i.e., with respect to a probability measure $$dP$$, and as a Riemann Integral where I would integrate with respect to $$dx$$. Do I somehow have to split the probability measure up into a conditional and a marginal one?

• That answer is correct. There will not be a joint density (either with respect to Lebesgue or counting measure) of a discrete and continuous random variable. Commented May 28, 2021 at 23:33
• for example, if $X$ is continuous and $Y$ is discrete, then consider the support of $(X, Y)$: because $Y$ is discrete and thus takes almost surely countably many values, the support of $(X, Y)$ is a Lebesgue-null set in $\mathbb{R}^2$. Therefore there is no joint density with respect to Lebesgue measure on $\mathbb{R}^2$. Similarly, the support of $(X, Y)$ is not countable because $X$ is continuous, so $(X, Y)$ cannot have a joint density with respect to counting measure Commented Jun 1, 2021 at 15:33
• Yes, at least not with respect to Lebesgue or counting measure on the product space. You can try to define things like the mixed joint density (which isn't a density with respect to Lebesgue or counting measure, but with respect to the product of the marginal dominating measures), but in my opinion that's just obscuring the conditioning that's going on behind the scenes Commented Jun 1, 2021 at 16:01
• yes, that’s what I meant in my last comment by “mixed joint density” Commented Jun 14, 2021 at 15:09
• I meant that in most cases, when you want to know the "mixed joint density" of two random variables, you usually need to know the conditional distribution of one given the other. In that case you'll arrive at formula $f_{X,Y}(x,y)=f_X(x)f_{Y\mid X}(y\mid x)$. So in a way, you don't need to know the mixed joint density at all. The important part is the conditional distribution. Commented Jun 14, 2021 at 17:12

This can be thought of as a companion answer to my answer to your related question about expectations with respect to joint distributions.

Suppose $$X$$ and $$Y$$ are real-valued random variables defined on a probability space $$(\Omega, \mathcal{A}, \mathbb{P})$$, with $$X$$ absolutely continuous with respect to Lebesgue measure and $$Y$$ discrete. Let $$\mathbb{P}_{X, Y}$$ be their joint distribution.

Then the general formula for the expectation of $$f(X, Y)$$ will be $$\mathbb{E}[f(X, Y)] = \int_{\mathbb{R} \times \mathbb{R}} f(x, y) \, \mathbb{P}_{X, Y}(d(x, y))$$ by either the law of the unconscious statistician or the change of variables formula for pushforward measures, or however else you want to call it. This formula uses neither of the assumptions on $$X$$ and $$Y$$ (it only assumes that the expectation exists).

Alternatively, the law of iterated expectation, as mentioned in the other answer, can be used to yield either $$\mathbb{E}[f(X, Y)] = \mathbb{E}[\mathbb{E}[f(X, Y) \mid X]]$$ or $$\mathbb{E}[f(X, Y)] = \mathbb{E}[\mathbb{E}[f(X, Y) \mid Y]].$$ Again, this does not use the assumptions on $$X$$ and $$Y$$, only the existence of the expectation. In practice, one of these two forms might be easier to compute than the other, so it's up to you to choose which one to use.

In both cases, you'll probably want to know a conditional distribution of one of the variables with respect to the other, which I'll go over next.

1. Suppose $$\mathbb{P}_{X \mid Y}$$ is a conditional distribution of $$X$$ given $$Y$$. Then we could write \begin{aligned} \mathbb{E}[f(X, Y)] &= \sum_{y \in \mathbb{R}} \mathbb{E}[f(X, Y) \mid Y = y] \mathbb{P}(Y = y) \\ &= \sum_{y \in \mathbb{R}} \left(\int_{\mathbb{R}} f(x, y) \, \mathbb{P}_{X \mid Y}(dx, y)\right) \mathbb{P}(Y = y). \end{aligned} The question then becomes how to compute $$\mathbb{P}_{X \mid Y}$$, and this depends on what you know about $$X$$ and $$Y$$ to begin with. However, one potential starting point is that this conditional distribution is determined by the condition $$\mathbb{P}(X \in B, Y \in C) = \sum_{y \in C} \left(\int_B \, \mathbb{P}_{X \mid Y}(dx, y)\right) \mathbb{P}(Y = y)$$ for all Borel sets $$B, C \subseteq \mathbb{R}$$.

It might be the case that you can compute a conditional density $$p_{X \mid Y}$$ of $$X$$ given $$Y$$ with respect to Lebesgue measure, in which case we would have $$\int_B \mathbb{P}_{X \mid Y}(dx, y) = \int_B p_{X \mid Y}(x, y) \, dx,$$ and hence $$\mathbb{E}[f(X, Y)] = \sum_{y \in \mathbb{R}} \left(\int_{\mathbb{R}} f(x, y) p_{X \mid Y}(x, y) \, dx\right) \mathbb{P}(Y = y).$$

2. Now suppose $$\mathbb{P}_{Y \mid X}$$ is a conditional distribution of $$Y$$ given $$X$$, and $$p_X$$ is the density of $$X$$ with respect to Lebesgue measure. In this case, $$\mathbb{E}[f(X, Y)] = \int_{\mathbb{R}} \left(\int_{\mathbb{R}} f(x, y) \, \mathbb{P}_{Y \mid X}(d y, x)\right) p_X(x) \, dx.$$ Again, being able to compute $$\mathbb{P}_{Y \mid X}$$ requires you to know something about $$X$$ and $$Y$$ beforehand, but it is determined by the condition $$\mathbb{P}(X \in B, Y \in C) = \int_B \left(\int_C \, \mathbb{P}_{Y \mid X}(dy, x)\right) p_X(x) \, dx$$ for all Borel sets $$B, C \subseteq \mathbb{R}$$. In this case, you can compute a conditional probability mass function $$p_{Y \mid X}$$ of $$Y$$ given $$X$$ (i.e., a conditional density with respect to counting measure) explictly by $$p_{Y \mid X}(y, x) = \mathbb{P}_{Y \mid X} (\{y\}, x) = \text{"}\mathbb{P}(Y = y \mid X = x)\text{"}.$$ and hence $$\mathbb{E}[f(X, Y)] = \int_{\mathbb{R}} \left(\sum_{y \in \mathbb{R}} f(x, y) \, p_{Y \mid X}(y, x)\right) p_X(x) \, dx.$$

• @guest1 a joint probability always exists: it's defined (using the notation I used) by $\mathbb{P}_{X, Y}(E) = \mathbb{P}((X, Y) \in E)$. And sorry if it caused confusion, but the $f$ that I used here is not a joint density but just some function whose expected value we want to compute Commented Jun 1, 2021 at 15:34
• @guest1 No need to apologize :). The integral is with respect to $\mathbb{P}_{Y\mid X}(dy, x)$, which is a discrete measure (i.e., absolutely continuous with respect to counting measure). Integrals over such measures are the same as sums. In other words, any countable sum $\sum_{i\in I} a_i$ is the same as the integral $\int_I a_i \, \mu(di)$ where $\mu$ is counting measure. I chose to present the integral because it is a formula that holds generally, not just in the discrete case. Commented Oct 4, 2021 at 4:45
• Yes to both. To the second point, every discrete probability distribution has a probability mass function, and that probability mass function is a density with respect to counting measure. Commented Oct 4, 2021 at 16:44
• Most books on measure-theoretic probability should cover the material similarly. Personally, I like Probability Theory: A Comprehensive Course by Klenke and A Modern Approach to Probability Theory by Fristedt and Gray. For your second question, yes: the formula I gave is a kind of Fubini-Tonelli formula. There are more complicated analogs for non-product sets Commented Jul 10, 2023 at 21:05
• Those kinds of formulas can be handled by various disintegration theorems: en.wikipedia.org/wiki/Disintegration_theorem Commented Jul 11, 2023 at 16:48

I am using slightly different notation that the OP in this answer.

This is a situation where the law of iterated expectation is very useful. Let $$Z=g(X,Y)$$ denote the random variable whose expectation we wish to compute. Then, the Law of Iterated Expectation tells us that

$$E[Z] = E[E[Z\mid Y]]$$

where $$E[Z\mid Y]$$ is a random variable that happens to be a function of the discrete random variable $$Y$$ that takes on values $$\{y_1, y_2, \cdots\}$$. Thus, \begin{align}E[Z] &= \sum_i P(Y=y_i)E[Z\mid Y=y_i]\\ &= \sum_i P(Y=y_i) E[g(X,Y)\mid Y =y_i]\tag{1}\\ \end{align}

But, given that the value of $$Y$$ is $$y_i$$, $$g(X,Y) = g(X,y_i)$$ depends on $$X$$ alone, not on $$Y$$ whose value is fixed at $$y_i$$. Consequently, we can compute $$E[g(X,Y)\mid Y =y_i]$$ using $$f_{X\mid Y=y}(x \mid Y=y)$$, the conditional density of $$X$$ given that $$Y=y$$. Well, let $$f(x,y)$$ be a nonnegative function with support $$\mathbb R\times \{y_1, y_2, \cdots\}$$ such that for each $$y_i$$, $$\int_\mathbb R f(x,y_i)\,\mathrm dx = P(Y = y_i)$$, or equivalently,

$$f_{X\mid Y=y_i}(x \mid Y=y_i) = \dfrac{f_{X,Y}(x,y_i)}{P(Y=y_i)}.$$

Similarly, the conditional mass function of $$Y$$ given that $$X = x$$ is

$$p_{Y \mid X = x]}(y_i) = \dfrac{f(x,y_i)}{\sum_j f(x,y_j).}$$

Note that $$f(x,y)$$ is not the joint density function of $$X$$ and $$Y$$; $$X$$ and $$Y$$ are not jointly continuous random variables and do not enjoy a joint density function. Then, $$(1)$$ simplifies to

\begin{align} E[g(X,Y)] &= \sum_i P(Y=y_i) \int_{-\infty}^\infty g(x,y_i)\dfrac{f_{X,Y}(x,y_i)}{P(Y=y_i)} \,\mathrm dx \\ &= \sum_i \int_{-\infty}^\infty g(x,y_i) f_{X,Y}(x,y_i) \,\mathrm dx.\tag{2}\end{align}

I have no idea how to write $$(2)$$ in terms of of an integral with respect to the probability measure $$P$$ as in $$\int \cdots\mathrm dP$$.

• Thank you for your answer! In the last line you probably mean the conditional density right? Also how would I write this in terms of an integral w.r.t. to the probability measure P, i.e., as something like $\int ... dP$? Commented May 21, 2021 at 8:26
• @guest1 See revised answer. Commented May 21, 2021 at 15:52