Apparent paradox in student T-test Let's say I have 2 series of data:

*

*series A has 1M samples, out of which 1000 are "1", the rest is 0

*series B has 100 samples, all 0s

I'm trying to use a statistical test to check if boths series have same mean or not.
To me, these series could have the same mean (ie 0.001).
Using scipy:
from scipy import stats
A = [0.0]*100
B = [1.0]*1000 + [0.0]*(1000000-1000)
scipy.stats.ttest_ind(A,B, equal_var=False)

Returns a  pvalue=1.399064173683964e-219, which means that both series are different.
It seems that the size of A is not taken into account, ie I get the same results if I switch A to be 1 million zeroes.
Is this expected?
To me, this p-value is inacurate. Have I made an incorrect assumption somewhere?
 A: Student's t-test is based on the normality of the data. You have manually generated binary data and I think it cannot be assumed they are normal.
You should test the probability of being 0 or 1, instead:
$$H_0: p_A = p_B, \text{ } H_A:p_a \neq p_B$$
You can estimate the probability of being 1 in the sample A and B as
$$\hat{p}_A = \frac{\text{number of ones}}{\text{sample size}} = 0$$
$$\hat{p}_B = \frac{\text{number of ones}}{\text{sample size}} = 0.001$$
Using the Moivre-Laplace central limit theorem you get
$$\frac{\hat{p}_A - \hat{p}_B}{\sqrt{\frac{\hat{p}_A (1 - \hat{p}_A)}{n_A} + \frac{\hat{p}_B (1 - \hat{p}_B)}{n_B}}}$$
is approximately $N(0,1)$ standard normal ($n_A$ and $n_B$ are the corresponding sample sizes).
So calculating the above test statistic, if it is greater than $1.96$ or less than $-1.96$, you should reject $H_0$ at significance level $0.05$ (the calculation left as an exercise to the reader :-) ).
A: Assuming $$H_0: \mu_1 = \mu_2, H_A: \mu_1 \neq \mu_2 $$ and $\alpha = 0.05$.
The t-statistics is calculated by:
$$t_{obs} = \frac{\hat{\mu_1}- \hat{\mu_2}}{\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}}$$
In your example, let's calculate the variances:
$$\sigma^2_1 = p(1-p)=0.001(0.999) = 0.000999$$
$$\sigma^2_2 = 0$$
Therefore, the t-statistic would be:
$$t_{obs} = \frac{0.001- 0}{\sqrt{\frac{0.000999}{1000000}+\frac{0}{100}}} \approx 31.64$$
(Notice that, because you know the variance, this is actually a z-test because $s^2 = \sigma^2$).
Because you have a two-sided test, your critical region is:
$$CR = [t < -1.96; t>1.96]$$
Therefore, you reject the null hypothesis and you can't say that the means are equal.
