If I understand your question and assumptions, then the answer is yes. This does not require any independence, but it does require that your conditional expectations are zero for all values of the auxiliary variables.
To show this mathematically, I will start with your first case (single auxiliary variable), then build to the more general case.
In the first instance, you have a variable of interest $x$, and a "nuisance" variable $a$. The expectation of $x$ is defined as
$$\langle x\rangle=\int p[x] x dx$$
where $p[x]$ is the probability density function (PDF) of $x$.
Given the joint PDF $p[x,a]$ of $x$ and $a$, the conditional PDF of $x$ given $a$ is
$$p[x|a]=\frac{p[x,a]}{p[a]}$$
so the conditional expectation of $x$ given $a$ is
$$\langle x|a\rangle=\frac{1}{p[a]}\int p[x,a] x dx$$
Expressing the marginal PDF for $x$ as
$$p[x]=\int p[x,a]da$$
and substituting into the $\langle x\rangle$ equation, we then have
$$\langle x\rangle=\int \langle x|a\rangle p[a] da$$
From this equation it follows that if $\langle x|a \rangle=0$ (for all $a$) then $\langle x\rangle=0$.
(Note that this derivation has not assumed independence, i.e. there is no need to have $p[x,a]=p[x]p[a]$.)
To generalize the result, it suffices to consider the case with two auxiliary variables $a$ and $b$. In this case, following along the path taken above, we have conditional probability
$$p[x|a,b]=\frac{p[x,a,b]}{p[a,b]}$$
conditional expectation
$$\langle x|a,b\rangle=\frac{1}{p[a,b]}\int p[x,a,b] x dx$$
and marginal (joint) probability
$$p[x,a]=\int p[x,a,b]db$$
Combining these we then have
$$\langle x|a\rangle=\frac{1}{p[a]}\int \langle x|a,b\rangle p[a,b] db$$
So if $\langle x|a,b \rangle=0$ (for all $a$ and $b$) then $\langle x|a\rangle=0$ for all $a$. (And from the first case, we then have $\langle x\rangle=0$.)
