Recall the abstract definitions of random variables.
### Sample spaces and probability
First, the setting. There is a "sample space" $\Omega$ with generic elements $\omega$. Certain subsets of $\Omega$ are distinguished: they are the ones to which we are willing to assign probabilities; they are called "events" (or "measurable sets"). Let $\mathbb{P}$ denote the probability function. Conceptually, it's a simple and natural thing: it assigns numbers between $0$ and $1$ to all the events, according to the axioms of probability.
### Random variables
A random variable $X$ assigns a definite number, written $X(\omega)$, to each $\omega\in\Omega$. A pair of random variables $X$ and $Y$ can be thought of as assigning ordered pairs of numbers, written $(X(\omega), Y(\omega))$, to each $\omega$.
New random variables may be constructed out of existing ones through formulas (as well as in other ways). For instance, $Z=XY$ can be thought of as extending the ordered pairs into ordered triples
$$(X(\omega), Y(\omega), Z(\omega)) = (X(\omega), Y(\omega), X(\omega)Y(\omega)).$$
That is, $Z=XY$ means you take the *definite numerical* values of $X$ and $Y$ at each outcome $\omega$ and multiply them *as numbers.*
There is a technical condition applied to random variables $X$, called "measurability." It means that given any number $x$, the set of all outcomes $\omega$ for which the value of $X$ is $x$ or less must be an event (thereby allowing us to find its probability). For much practical and conceptual work we may ignore this technicality. It guarantees that the distribution functions $F_X$, to be defined below, always exist.
### Expectation
The expectation operator generalizes the idea of weighting all the values of a random variable by their probability and summing that up to get an "average" value. It is written
$$\mathbb{E}(X) = \int_\Omega X(\omega)\mathrm{d}\mathbb{P}(\omega).\tag{1}$$
The notation is intended to remind us of the key ideas: $\int$ (which looks like a stretched out "S", for "sum") refers to the generalization of summation; $\Omega$ references the sample space; $X(\omega)$ are the values of $X$; and $d\mathbb{P}(\omega)$ are the associated probabilities. There is a "$\mathrm{d}$" in front of it as a reminder that individual observations $\omega$ might not actually have probabilities, so some special steps have to be taken in general to make sense of this.
### Law of the Unconscious Statistician
It is rare to use $(1)$ for computation. Instead, we derive an intermediate mathematical object called the *distribution* of $X$, often written $F_X$ (even though it depends on $\mathbb{P}$ as well). It simply is the chance of the event $X \le x$:
$$F_X(x) = \mathbb{P}\left(\{\omega\in\Omega\mid X(\omega) \le x\}\right).$$
The Law of the Unconscious Statistician (LOTUS) is a theorem that says the expectation $(1)$ can be computed by integrating $F$:
$$\mathbb{E}(X) = \int_\mathbb{R} x dF_X(x).\tag{2}$$
This is no longer an abstract integral: it's a more familiar object from elementary Calculus. In general it needs to be a Lebesgues-Stieltjes integral in order to handle discrete probability distributions, but it's perfectly fine to think of it as either one of two things:
* When $F_X$ is *discrete* with probability $p_X(x)$ that $X=x$ for at most countably many distinct $x$, $(2)$ is the sum $$\mathbb{E}(X) = \sum_x x p_X(x).$$ It is defined only when that sum has the same value regardless of the order in which the $x$ are summed.
* When $F_X$ is *absolutely continuous* with continuous probability density $f_X(x) = F^\prime_X(x)$, it is the Riemann integral $$\mathbb{E}(X) = \int_{-\infty}^\infty x f(x) dx.$$ It is defined only when this integral converges (it's a double limit as the right and left endpoints go to $\pm\infty$).
When there are multiple random variables, the distribution is defined similarly. For instance,
$$F_{(X,Y)}(x,y) = \mathbb{P}\left(\{\omega\in\Omega\mid X(\omega)\le x, Y(\omega) \le y\}\right)$$
for all numbers $x$ and $y$. The LOTUS now involves two-dimensional real integrals, frequently expressed as Riemann or Lebesgue integrals. The concept of expectation needs to be generalized slightly, though, because the ordered pair $(X,Y)$ is never just a number. Instead, let $h$ be a function of two real variables. Provided that $h$ is measurable and the chance of $h(X,Y)$ being undefined is zero (which permits, among other things, the possibility of analyzing functions like $h(x,y)=x/y$ which are not defined when $y=0$), $h(X,Y)$ is another random variable. This is meant in exactly the same sense $Z=XY$ is a random variable: it's a way of combining the numbers $X(\omega)$ and $Y(\omega)$ into a third number $h(X(\omega), Y(\omega))$ for each $\omega$ (throwing away any $\omega$ for which this combination is undefined, *provided* the set of such $\omega$ has zero probability). Now
$$\mathbb{E}(h(X,Y)) = \int_\mathbb{R}\int_\mathbb{R} h(x,y) \frac{\partial^2}{\partial x\partial y}F(x,y)\mathrm{d}x\mathrm{d}y$$
(with a double sum appearing instead of a double integral for discrete distributions). The function $\frac{\partial^2}{\partial x\partial y}F(x,y)$ (if it exists) is called the *bivariate density* of $(X,Y)$.
### Answers
**Let's use the LOTUS to resolve the questions.** Assume $F_{X,Y}$ has a bivariate density $\frac{\partial^2}{\partial x\partial y}F(x,y) = q(x,y)$ and that $F_Z$ has a density $p(z)$.
1. $\mathbb{E}(Z)$ (there's no need for a subscript) must refer to an expression like $(1)$; namely, $$\mathbb{E}(Z) = \int_\Omega Z(\omega)d\mathbb{P}(\omega).$$ LOTUS asserts this can be written as an "ordinary" integral, $$\mathbb{E}(Z) = \int_\Omega z\,p(z)dz.$$ The (bivariate) LOTUS, applied to $Z$ in in terms of $(X,Y)$, says this can be written $$\mathbb{E}(Z)= \mathbb{E}(XY)= \int_\Omega X(\omega)Y(\omega)\mathrm{d}\mathbb{P}(\omega)= \int_{\mathbb{R}^2} x y\, q(x,y)\mathrm{d}x\mathrm{d}y.$$
2. $\mathbb{E}_{X,Y}(XY)$ asks for the expectation of $XY=Z$ while, via the subscripts, making it explicitly clear that *both $X$ and $Y$ are considered random variables*. (Alternatively, we might fix the value of one of them and take an expectation with respect to the other: those are the *marginal expectations.*) Thus $$\mathbb{E}_{X,Y}(XY) = \int_\Omega X(\omega)Y(\omega)d\mathbb{P}(\omega).$$ Clearly this is *exactly* the same thing as $\mathbb{E}(Z)$ (above).
3. Expressions like "$\mathbb{E}_Z(XY)$" and "$\mathbb{E}_{XY}(Z)$" don't seem to make much sense. If they were to appear somewhere, I would first attempt to understand them both as $\mathbb{E}(Z)$.
4. We can use LOTUS in reverse. Take, for instance, the expression $$\int_{\mathbb{R}} z p(z) \mathrm{d}z$$ from the question. That sure looks like an expectation. Here's what happens: $Z$ itself is a random variable. It therefore has a distribution $F_Z(z)$ defined in the usual way. Let this distribution have a density $p$. LOTUS asserts $$\mathbb{E}(Z) = \int_\mathbb{R} z p(z) \mathrm{d}z.$$ Comparing this to the foregoing shows that $$\int_\mathbb{R} z p(z) \mathrm{d}z = \mathbb{E}(Z) = \int_{\mathbb{R}^2} x y\, q(x,y)\mathrm{d}x\mathrm{d}y.$$
5. The integral $(c)$ in the question is not an expectation and $(d)$ and $(e)$ are not well-defined because they involve undefined, uninstantiated variables. (For instance, $(d)$ involves $x$ and $y$ which aren't integrated out.)
----
I relied implicitly on many resources in writing this very brief account of the notation. One of the very best and most accessible introductions is
Steven Shreve, *Stochastic Calculus for Finance II: Continuous-Time Models*, Chapter 1. Springer, 2004.
I also had in mind some of the concepts (especially concerning bivariate distributions) introduced in
Roger Nelsen, *An Introduction to Copulas*, 2nd Ed., Chapter 2. Springer, 2006.