Test of difference in proportions with non-random/self-selecting samples Suppose I have two versions of an email and each contains a link. I randomly assign two groups to receive the respective emails of sizes $N_1$ and $N_2$ (i.e. group $i$ receives email $i$ for $i=1,2$). Denote the number of group $i$ recipients who open the email as $n_i$ for $i=1,2$. Further, denote the number of group $i$ recipients who clicked the link after opening the emails in the respective groups as $c_i$ for $i=1,2$. I am interested in whether the proportion $\frac{c_i}{n_i}$ is different for the two emails/groups. Two questions:

*

*Can I apply a standard $z$ test given that the samples are not random, i.e. recipients self-select by opening the emails.

*If so, my understanding is that I would be using $n_1$ and $n_2$ as sample sizes. Is that correct?

I have tried to search for answers online but am not really sure how to concisely articulate this question. Any journal articles or textbook references would be very helpful. Thanks in advance!
 A: You might find it useful (and relevant) to model this situation by supposing that among people who open these emails, (1) the decisions to click through are independent and (2) those decisions are made with fixed probabilities $p_i$ which might differ by type of email.
The first assumption is non-controversial (unless you are sending emails in blocks to groups of related people).  The second one ought to be tested, but we can at least use it to proceed with some analysis.
I also assume the recipients are blind as to the types of emails: that is, superficially they appear identical, so that the decision to open the email is independent of its type.
The randomized choice of recipients of these emails assures that you have random samples of two (overlapping) populations: namely, those who would open emails of type 1 and those who would open emails of type 2.  Assuming these populations are large compared to the numbers of emails sent, the lack of independence in the sampling (nobody has any chance of receiving both types of emails) is inconsequential.
These assumptions place you in the textbook situation of comparing two (hypothetical) probabilities $p_i$ in two populations based on a two-sample Binomial experiment with sample sizes $n_1$ and $n_2,$ thereby confirming both assertions (1) and (2) in the question.
