Does this expression hold? $$\max_{x\in X}E[Y\mid X=x] = \max_{P_X}\sum_{x\in X} P_X(x)E[Y\mid X=x]$$ Left side says: max over all available $x$ for the expected value of $Y$ given $X=x$.

Right side says: maximize over probabilities distribution of $X$

I can't seem to figure it out. I think that ifyou wan't to max over probablity distribution on the right hand side you would eventually maximize over $E[Y\mid X=x]$ and say you get that maximization is for $x=x'$ then you would simply give $P_X(x')=1$ and then the expressions are equal. Am I right? How can you prove this? Thanks

  • 2
    $\begingroup$ This is a bizarre usage of notation, because you're treating $X$ as both a random variable and a state space. $\endgroup$ – Alex R. Oct 25 '17 at 17:23
  • $\begingroup$ say $x \in {0,1}$ is random variable, for x=0 you have a value for $E{[Y|x=0]}$ and for x=1 you have a value for $E{[Y|x=1]}$. now say you can choose the probability $P_x(0)$ and $P_x(1)$. $\endgroup$ – user3921 Oct 25 '17 at 17:24

Let me call the set in which $X$ takes values $\mathcal X$ and define $h(x) = E(Y\mid X = x)$ for cleaner notation [I take it as given that such a $h$ exists].

Your question statement seems to imply that $h$ attains its supremum on $\mathcal X$: $h(x^\star) = \sup_{x\in \mathcal X}h(x)$ for some $x^\star \in \mathcal X$.

Now for any variable $X$ with distribution $P_X$ on $\mathcal X$:

$$ E(h(X)) = \int_{\mathcal X} h(x)P_X(dx) \leq \int_{\mathcal X}h(x^\star)P_X(dx) = h(x^\star).$$

But if $X$ takes the value $x^\star$ with probability one, then the inequality is an equality.


If state space is discrete (as your notation implies) the $\max$ exists (e.g. because the state space is finite) then the expression holds almost trivially- your argument says the RHS attains the LHS. Moreover, it cannot exceed the left because it is less than $$ \max_{x\in\mathcal{S}} \mathbb{E}[Y | X = x] \max_{p \in \mathcal{P}(\mathcal{S}) }\sum_{x\in\mathcal{S}} p(x), $$ which is the LHS. (As @Alex R. said your original notation is bizarre and I took the liberty of denoting by $\mathcal{P}(\mathcal{S})$ a suitable space of probability measures on the state space $\mathcal{S}$.

If you are actually talking about $\sup$ (still assuming the space is discrete) then you can take a sequence of $x^\star_n$ such that $$ \mathbb{E}[Y | X = x^\star_n] \to \sup_{x\in\mathcal{S}} \mathbb{E}[Y | X = x] $$ and the argument still works.

Moreover, if your space is continuous and your space $\mathcal{P}{(\mathcal{S})}$ includes discrete measures, again the same argument works but you need integrals instead of the sums etc. If you only want continuous distributions then you need to work a little harder to approximate $\delta$-measures.

So basically, your inequality works with $\max$ replaced by $\sup$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.