Assume I only did a completely randomized experiment (CRE), i.e., $N$ subjects, I randomly assign $N_t$ (say $N_t=N/2$) of them to the treatment group and the rest to the control group. Now, after I finish the experiment and try to compute the average treatment effect. I also see that I recorded the gender of each subject. Now assume my sample size is large, i.e., $N$ is big, then as the randomization is successful, I find that within each group, the gender is well-balanced, i.e., it "looks like" I did a blocking first and randomized second (stratified randomized experiment (SRE)). Now my question is the following:
Let $Y_i(0)$ be the potential outcome of subject $i$ if he is assigned to control group and $Y_i(1)$ be the potential outcome of subject $i$ if he is assigned to treatment group. Our goal is to get $\tau_{fs}=\frac{1}{N}\sum_{i=1}^N[Y_i(1)-Y_i(0)]$. This is impossible to directly get, as for each subject, we only observe one of his potential outcome. Now here I have 2 estimator to estimate this quantity $\tau_{fs}$. Let $W_i\in\{0,1\}$ denote whether subject $i$ is assigned to control ($W_i=0$) or treatment group ($W_i=1$) and let $X_i\in\{m,f\}$ be the subject $i$'s gender. Let $Y^{obs}_i=Y_i(W_i)$ be the observed value of subject $i$. Let $N_t$ and $N_c$ be the total number of people in the treatment and control group
\begin{equation} \hat{\tau}_1=\frac{1}{N_t}\sum_{i:W_i=1}Y^{obs}_i-\frac{1}{N_c}\sum_{i:W_i=0}Y^{obs}_i \end{equation} This is just take the average within each group, then take the difference. Here is a more complex one, let $N_f$ and $N_m$ be the total number of female and male in the sample. \begin{equation} \hat{\tau}_2=\frac{N_f}{N}\hat{\tau}_f+\frac{N_m}{N}\hat{\tau}_m \end{equation} where \begin{equation} \hat{\tau}_x=\frac{1}{N_t(x)}\sum_{i:X_i=x,W_i=1}Y^{obs}_i-\frac{1}{N_c(x)}\sum_{i:X_i=x,W_i=0}Y^{obs}_i,~\mbox{with }x\in\{m,f\} \end{equation} Namely, this estimator, "pretend" that we did a stratified random experiment, so it first take the treatment effect within each category based on the gender, then it did a weighted average of these two treatment effects and weight is the fraction of male (and female) in the sample. One can show that both $\hat{\tau}_1$ and $\hat{\tau}_2$ are unbiased estimator of $\tau_{fs}$.
I did a simulation study, I found that $\hat{\tau}_2$ seems always better than $\hat{\tau}_1$, I wonder if there is any mathematical or intuitive justification for that? Also, one can show that the variance of $\hat{\tau}_2$ is less than $\hat{\tau}_1$. So in practice is there a reason NOT to use $\hat{\tau}_2$ but just use $\hat{\tau}_1$? Will this weighed average $\hat{\tau}_2$ lead to something like Simpson's paradox?
As stratified random experiment requires extra work than completely randomized experiment, in practice, IF $N$ IS BIG, should we just do a completely randomized experiment and in the data analysis part, we just use $\hat{\tau}_2$ and its variance, i.e., "pretend" the data comes from stratified random experiment? My justification is that as $N$ is big, so the attributes will be balanced within each group (and one can check it empirically). I understand that when $N$ is small, then just a completely randomized experiment without stratified would lead you to the situation like all male in the treatment group and all female in the control group, then the result is useless. So when $N$ is small, then stratified random experiment eliminates this possibility. But other than that, I don't see a point of doing stratified random experiment, especially considering that it requires extra work than just completely randomized experiment. Did I miss something important?
The $\hat{\tau}_2$ estimator has lower variance than $\hat{\tau}_1$. However, when analyzing the data, we could also run a linear regression with the treatment variable and the gender attributes variable, then we can derive the ols estimator for the treatment effect (the coefficient in front of the treatment variable) as $\hat{\tau}_3$. Note that the variance of $\hat{\tau}_3$ is also lower than that of $\hat{\tau}_1$. But $\hat{\tau}_3$ and $\hat{\tau}_2$ are equal. Which one should we use?
Specifically, one can run the following regression model \begin{equation} Y_i = \alpha' + \tau'_3W_i+\epsilon' \end{equation} It can be shown that the the ols estimator of the above model denoted by $\hat{\tau}'_3=\hat{\tau}_1$.
One can also run another regression model with covariates (gender) included \begin{equation} Y_i = \alpha + \tau_3W_i+\beta X_i+\epsilon \end{equation} It can be shown that the ols estimator $\hat{\tau}_3$ is consistent for the true treatment effect and $\hat{\tau}_3$ has less variance than $\hat{\tau}_1$.
In addition, for $\hat{\tau}_2$, it in the form of weighted average. But I also see some people use different weight than the gender proportion. Instead, they use inverse of variance within each group as the weight. But in that case will $\hat{\tau}_2$ still be a unbiased estimator of $\tau_{fs}$?
