Correctly using hypothesis testing to conclude X mean is less than Y mean I have two unknown distributions $A$ and $B$. I drew samples from them and store them in array a and b, respectively. I want to show that $E[A]$ is likely less than $E[B]$ (forgive my informal language, my statistics is too rusty) with a confidence level of 95%.
For this, I'm testing two null hypotheses with t-tests using SciPy:

*

*$H_0: E[A] = E[B]$. The p-value returned by scipy.stats.ttest_ind(a, b, equal_var=False, alternative='two-sided') is p1.

*$H_0: E[A] > E[B]$. The p-value returned by scipy.stats.ttest_ind(a, b, equal_var=False, alternative='greater') is p2.

What conditions must p1 and p2 satisfy for me to make the aforementioned conclusion? I know they must be small enough for me to reject these null hypotheses, but I'm not sure what values exactly to use.
Also, is this method even correct in the first place? I believe I can't make the conclusion by showing I fail to reject $H_0: E[A] < E[B]$, but I'm not sure whether the reverse way I described above is sound, either.
 A: You are overthinking the problem a bit.
Based on your description, you have two samples from populations $A$ and $B$ where you want to show $\mathbb{E}(A) < \mathbb{E}(B)$. Assuming you drew enough samples and the populations do not follow distributions that are extremely skewed, you plan to use a $t$-test which is practically applicable.
A standard (read: conventional) way to do so is to specify
$$ H_0: \mathbb{E}(A) = \mathbb{E}(B),\quad H_1: \mathbb{E}(A) < \mathbb{E}(B)$$
and run on your Python:
scipy.stats.ttest_ind(a, b, equal_var=False, alternative='less')
(Note equal_var=False is copied directly from the question - you should consider whether this is actually the case, as it changes the result a bit).
Why your proposals do not work

$H_0: \mathbb{E}(A) = \mathbb{E}(B)$. The p-value returned by scipy.stats.ttest_ind(a, b, equal_var=False, alternative='two-sided')

Here you are testing against a two-sided alternate hypothesis $H_a: \mathbb{E}(A) \neq \mathbb{E}(B)$. Rejecting the null hypothesis in this case technically tells you nothing on which expectation is greater than which.
Moreover, from a practical perspective the p-value you get from the same samples will be double than that you get if you use $H_1$ as your alternate hypothesis, as you are also required to consider the other extreme in this hypothesis.

$H_0: \mathbb{E}(A) > \mathbb{E}(B)$. The p-value returned by scipy.stats.ttest_ind(a, b, equal_var=False, alternative='greater')

Firstly, the Python code you've suggested does not match the null hypothesis you have written, but the following
$$H_0: \mathbb{E}(A) = \mathbb{E}(B),\quad H_b:  \mathbb{E}(A) > \mathbb{E}(B)$$
This is confirmed on the documentation for scipy.stats.ttest_ind, "This is a two-sided test for the null hypothesis that 2 independent samples have identical average (expected) values." there is no guarantee that the result (i.e. p-value) will extrapolate to the null of $\mathbb{E}(A) > \mathbb{E}(B)$.
Secondly, for such composite null hypothesis there is quite a lot of work involved to integrate the test statistic, p-value, etc. for every possible value in the set. As you can probably imagine, this gets pretty messy quick and I would not recommend doing so given the existence an easy alternative.
