Express expectation value of a joint distribution over a discrete and continuous random variable

Let $$Y$$ be a discrete random variable and let $$X$$ be an (absolutely) continuous random variable and $$f(X, Y)$$ a function of these two random variables. Let $$P(X, Y)$$ be the joint probability measure. I am now wondering how to properly write the joint expectation value $$\mathbb{E}[f(X, Y)]$$? I would write something like: $$\mathbb{E}[f(X, Y)] = \sum_{y} \int f(x, y) dP(X=x, Y=y),$$ but I think this cannot be quite right because I shouldnt integrate over y. So I would like to know how to write it down, both in terms of a Lebesgue integral, i.e., with respect to a probability measure $$dP$$, and as a Riemann Integral where I would integrate with respect to $$dx$$. Do I somehow have to split the probability measure up into a conditional and a marginal one?

• That answer is correct. There will not be a joint density (either with respect to Lebesgue or counting measure) of a discrete and continuous random variable. – Artem Mavrin May 28 at 23:33
• for example, if $X$ is continuous and $Y$ is discrete, then consider the support of $(X, Y)$: because $Y$ is discrete and thus takes almost surely countably many values, the support of $(X, Y)$ is a Lebesgue-null set in $\mathbb{R}^2$. Therefore there is no joint density with respect to Lebesgue measure on $\mathbb{R}^2$. Similarly, the support of $(X, Y)$ is not countable because $X$ is continuous, so $(X, Y)$ cannot have a joint density with respect to counting measure – Artem Mavrin Jun 1 at 15:33
• Yes, at least not with respect to Lebesgue or counting measure on the product space. You can try to define things like the mixed joint density (which isn't a density with respect to Lebesgue or counting measure, but with respect to the product of the marginal dominating measures), but in my opinion that's just obscuring the conditioning that's going on behind the scenes – Artem Mavrin Jun 1 at 16:01
• yes, that’s what I meant in my last comment by “mixed joint density” – Artem Mavrin Jun 14 at 15:09
• I meant that in most cases, when you want to know the "mixed joint density" of two random variables, you usually need to know the conditional distribution of one given the other. In that case you'll arrive at formula $f_{X,Y}(x,y)=f_X(x)f_{Y\mid X}(y\mid x)$. So in a way, you don't need to know the mixed joint density at all. The important part is the conditional distribution. – Artem Mavrin Jun 14 at 17:12

This can be thought of as a companion answer to my answer to your related question about expectations with respect to joint distributions.

Suppose $$X$$ and $$Y$$ are real-valued random variables defined on a probability space $$(\Omega, \mathcal{A}, \mathbb{P})$$, with $$X$$ absolutely continuous with respect to Lebesgue measure and $$Y$$ discrete. Let $$\mathbb{P}_{X, Y}$$ be their joint distribution.

Then the general formula for the expectation of $$f(X, Y)$$ will be $$\mathbb{E}[f(X, Y)] = \int_{\mathbb{R} \times \mathbb{R}} f(x, y) \, \mathbb{P}_{X, Y}(d(x, y))$$ by either the law of the unconscious statistician or the change of variables formula for pushforward measures, or however else you want to call it. This formula uses neither of the assumptions on $$X$$ and $$Y$$ (it only assumes that the expectation exists).

Alternatively, the law of iterated expectation, as mentioned in the other answer, can be used to yield either $$\mathbb{E}[f(X, Y)] = \mathbb{E}[\mathbb{E}[f(X, Y) \mid X]]$$ or $$\mathbb{E}[f(X, Y)] = \mathbb{E}[\mathbb{E}[f(X, Y) \mid Y]].$$ Again, this does not use the assumptions on $$X$$ and $$Y$$, only the existence of the expectation. In practice, one of these two forms might be easier to compute than the other, so it's up to you to choose which one to use.

In both cases, you'll probably want to know a conditional distribution of one of the variables with respect to the other, which I'll go over next.

1. Suppose $$\mathbb{P}_{X \mid Y}$$ is a conditional distribution of $$X$$ given $$Y$$. Then we could write \begin{aligned} \mathbb{E}[f(X, Y)] &= \sum_{y \in \mathbb{R}} \mathbb{E}[f(X, Y) \mid Y = y] \mathbb{P}(Y = y) \\ &= \sum_{y \in \mathbb{R}} \left(\int_{\mathbb{R}} f(x, y) \, \mathbb{P}_{X \mid Y}(dx, y)\right) \mathbb{P}(Y = y). \end{aligned} The question then becomes how to compute $$\mathbb{P}_{X \mid Y}$$, and this depends on what you know about $$X$$ and $$Y$$ to begin with. However, one potential starting point is that this conditional distribution is determined by the condition $$\mathbb{P}(X \in B, Y \in C) = \sum_{y \in C} \left(\int_B \, \mathbb{P}_{X \mid Y}(dx, y)\right) \mathbb{P}(Y = y)$$ for all Borel sets $$B, C \subseteq \mathbb{R}$$.

It might be the case that you can compute a conditional density $$p_{X \mid Y}$$ of $$X$$ given $$Y$$ with respect to Lebesgue measure, in which case we would have $$\int_B \mathbb{P}_{X \mid Y}(dx, y) = \int_B p_{X \mid Y}(x, y) \, dx,$$ and hence $$\mathbb{E}[f(X, Y)] = \sum_{y \in \mathbb{R}} \left(\int_{\mathbb{R}} f(x, y) p_{X \mid Y}(x, y) \, dx\right) \mathbb{P}(Y = y).$$

2. Now suppose $$\mathbb{P}_{Y \mid X}$$ is a conditional distribution of $$Y$$ given $$X$$, and $$p_X$$ is the density of $$X$$ with respect to Lebesgue measure. In this case, $$\mathbb{E}[f(X, Y)] = \int_{\mathbb{R}} \left(\int_{\mathbb{R}} f(x, y) \, \mathbb{P}_{Y \mid X}(d y, x)\right) p_X(x) \, dx.$$ Again, being able to compute $$\mathbb{P}_{Y \mid X}$$ requires you to know something about $$X$$ and $$Y$$ beforehand, but it is determined by the condition $$\mathbb{P}(X \in B, Y \in C) = \int_B \left(\int_C \, \mathbb{P}_{Y \mid X}(dy, x)\right) p_X(x) \, dx$$ for all Borel sets $$B, C \subseteq \mathbb{R}$$. In this case, you can compute a conditional probability mass function $$p_{Y \mid X}$$ of $$Y$$ given $$X$$ (i.e., a conditional density with respect to counting measure) explictly by $$p_{Y \mid X}(y, x) = \mathbb{P}_{Y \mid X} (\{y\}, x) = \text{"}\mathbb{P}(Y = y \mid X = x)\text{"}.$$ and hence $$\mathbb{E}[f(X, Y)] = \int_{\mathbb{R}} \left(\sum_{y \in \mathbb{R}} f(x, y) \, p_{Y \mid X}(y, x)\right) p_X(x) \, dx.$$

• Thank you for your great answer! Two questions: Does a joint probability measure always exist (even when one RV is continuous and the other discrete)? You are using the joint density $f(x, y)$ here but in your comment below my answer you are saying that such a density does not exist. How does this fit together? – guest1 Jun 1 at 10:02
• @guest1 a joint probability always exists: it's defined (using the notation I used) by $\mathbb{P}_{X, Y}(E) = \mathbb{P}((X, Y) \in E)$. And sorry if it caused confusion, but the $f$ that I used here is not a joint density but just some function whose expected value we want to compute – Artem Mavrin Jun 1 at 15:34
• Great thanks again! – guest1 Jun 1 at 15:40

I am using slightly different notation that the OP in this answer.

This is a situation where the law of iterated expectation is very useful. Let $$Z=g(X,Y)$$ denote the random variable whose expectation we wish to compute. Then, the Law of Iterated Expectation tells us that

$$E[Z] = E[E[Z\mid Y]]$$

where $$E[Z\mid Y]$$ is a random variable that happens to be a function of the discrete random variable $$Y$$ that takes on values $$\{y_1, y_2, \cdots\}$$. Thus, \begin{align}E[Z] &= \sum_i P(Y=y_i)E[Z\mid Y=y_i]\\ &= \sum_i P(Y=y_i) E[g(X,Y)\mid Y =y_i]\tag{1}\\ \end{align}

But, given that the value of $$Y$$ is $$y_i$$, $$g(X,Y) = g(X,y_i)$$ depends on $$X$$ alone, not on $$Y$$ whose value is fixed at $$y_i$$. Consequently, we can compute $$E[g(X,Y)\mid Y =y_i]$$ using $$f_{X\mid Y=y}(x \mid Y=y)$$, the conditional density of $$X$$ given that $$Y=y$$. Well, let $$f(x,y)$$ be a nonnegative function with support $$\mathbb R\times \{y_1, y_2, \cdots\}$$ such that for each $$y_i$$, $$\int_\mathbb R f(x,y_i)\,\mathrm dx = P(Y = y_i)$$, or equivalently,

$$f_{X\mid Y=y_i}(x \mid Y=y_i) = \dfrac{f_{X,Y}(x,y_i)}{P(Y=y_i)}.$$

Similarly, the conditional mass function of $$Y$$ given that $$X = x$$ is

$$p_{Y \mid X = x]}(y_i) = \dfrac{f(x,y_i)}{\sum_j f(x,y_j).}$$

Note that $$f(x,y)$$ is not the joint density function of $$X$$ and $$Y$$; $$X$$ and $$Y$$ are not jointly continuous random variables and do not enjoy a joint density function. Then, $$(1)$$ simplifies to

\begin{align} E[g(X,Y)] &= \sum_i P(Y=y_i) \int_{-\infty}^\infty g(x,y_i)\dfrac{f_{X,Y}(x,y_i)}{P(Y=y_i)} \,\mathrm dx \\ &= \sum_i \int_{-\infty}^\infty g(x,y_i) f_{X,Y}(x,y_i) \,\mathrm dx.\tag{2}\end{align}

I have no idea how to write $$(2)$$ in terms of of an integral with respect to the probability measure $$P$$ as in $$\int \cdots\mathrm dP$$.

• Thank you for your answer! In the last line you probably mean the conditional density right? Also how would I write this in terms of an integral w.r.t. to the probability measure P, i.e., as something like $\int ... dP$? – guest1 May 21 at 8:26
• @guest1 See revised answer. – Dilip Sarwate May 21 at 15:52