You can test equality of the mean parameters against the alternative that the mean parameters are unequal with a likelihood ratio test. (However, if the mean parameters do differ and the distribution is exponential, this is a scale shift, not a location shift.)
Let's say we parameterize the $i$th observation in the first exponential as having pdf $1/\mu_x \exp(-x_i/\mu_x)$ and the $j$th observation in the second sample as having pdf $1/\mu_y \exp(-y_j/\mu_y)$ (over the obvious domains for the observations and parameters).
(To be clear, we're working in the mean-form not the rate-form here; this won't affect the outcome of the calculations.)
Since the distribution of $X_i$ is a special case of the gamma, $\Gamma(1,\mu_x)$, the distribution of the sum of $X$'s, $S_x$ is distributed $\Gamma(n_x,\mu_x)$; similarly that for the sum of the $Y$s, $S_y$ is $\Gamma(n_y,\mu_y)$. Because of the relationship between gamma distributions and chi-squared distributions, it turns out that$2/\mu_x S_x$ is $\chi^2_{2n_x}$. Hence the ratio, $\frac{S_x/\mu_x}{S_y/\mu_y} \sim F_{2n_x,2n_y}$.
Under the null hypothesis of equality of means, then, $\frac{S_x}{S_y} = \frac{n_x \bar x}{n_y \bar y} \sim F_{2n_x,2n_y}$, and under the two sided alternative, the values might tend to be either smaller or larger than a value from the null distribution, so you need a two-tailed test.