Since my question did not receive an answer, I try to improve it and make it more explicit.
I would like to understand the reasoning behind this passage ( par. D.5, page 19, https://arxiv.org/pdf/1605.07723.pdf ). Here the passage I would like to understand (in their minds, it should be trivial ):
Here $x,y$ are r.v., $f,\lambda$ some discrete functions, and $w$ parameters.
The conditional independence assumption they are referring to is:
$y \perp f(x) | \lambda(x) $
but I was not able to use it to derive the third line from the second.
As far as I understand, in the second line the internal expectation is a conditional expectation over $y$ (since $x$ does not appear in the function, but only $\overline{x}$), where the distribution of $y$ is $p(y|x=\overline{x})$, whereas in the third line we still can consider it as an expectation over $y$ (since $x$ does not appear in the function), but under a different conditional distribution $p(y|\lambda(x)=\lambda(\overline{x}))$. I am not able at the moment to use the conditional independence assumption to derive the third line from the second. I think we should use that $p(y|f(x),\lambda(x))=p(y|\lambda(x))$, so that in some sense the distribution of $y$ does not depend on the value of $f(x)$, once we condition on $\lambda(x)$. This would suggest maybe to write things like $E_x \rightarrow E_{\lambda} E[...|\lambda]$ (symbolically), but I did not arrive to any conclusion.
Any help/suggestion ?
PREVIOUS/OLD VERSION OF THE QUESTION:
I am reading this article https://arxiv.org/pdf/1605.07723.pdf and I am confused about the step in the proof of lemma D.5 at page 19.
I try to make the argument abstract, so that it should not be mandatory to read the article in order to understand the question.
As far as I understand we have the setting:
1 $X, Y$ are random variables
[2] $f(X), \lambda(X)$ are deterministic functions of $X$ ( so themselves random variable)
[3] $Y \perp f(X) | \lambda(X) $
[4] $g=g(f(X),Y))$ is a deterministic function of $f(X),Y$. In the article they have $g(f(X),Y)=log(1+e^{-wf(X)Y})$, with $w$ parameters, but I am not sure the precise functional form matters.
The goal would be to exploit the properties to rewrite $E[g(f(X),Y)]$ in a different manner.
First they use something similar to the law of total expectation. Given $\overline{X},\overline{Y}$ distributed as $X,Y$ but independent w.r.t to them one can first write:
$E_{X,Y}[g(f(X),Y)]=E_{\overline{X},\overline{Y}}[E_{X,Y}[g(f(\overline{X}),Y)|X=\overline{X}]]$ [Eq 1]
Than they say:
$E_{X,Y}[g(f(X),Y)]=E_{\overline{X},\overline{Y}}[E_{X,Y}[g(f(\overline{X}),Y)|\lambda(X)=\lambda(\overline{X})]]$ [Eq 2]
I guess here they are using the partial independence hypothesis, that says that fixing $\lambda(X)$ to some value makes $Y$ and $f(X)$ independent, but I am not sure how exactly how to apply this. Should it be trivial how to do that ?
My question is how the authors managed to derive [Eq 2] starting from [Eq 1].
UPDATE:
Following some comments in the chat, an idea is to start writing:
$E_{\overline{X},\overline{Y}}[E_{X,Y}[g(f(\overline{X}),Y)|X=\overline{X}]]=E_{\overline{X},\overline{Y}}[E_{X,Y} [g(f(\overline{X}),Y)|X=\overline{X},\lambda(X)=\lambda(\overline{X})]]$
this is true because essentialy if $X=\overline{X}$ than $\lambda(X)=\lambda(\overline{X})$ and we are just adding redundant information. The big step of course is to get rid of the condition $X=\overline{X}$, which I have problems in doing it formally using the conditional independence condition [3].