I have two processes, lets call them A and B. I want to infer whether their population means are equal for their experimental outcomes. Both of these processes generate 1s and 0s only. That is, the outcome is binary.
My situation is similar to using two different processes to flip a coin a few times. The idea is to compare the experimental outcomes of these two processes.
I will show two simple cases to illustrate the problem. In the first case, function
t.test() of the R language's stats package runs just fine when t-testing these two outcomes:
> t.test(rep(1,5), c(rep(1,4),0)) Welch Two Sample t-test data: rep(1, 5) and c(rep(1, 4), 0) t = 1, df = 4, p-value = 0.3739 ...
IN the second case, for some reason the t.test will not run, and an error is returned, when t-testing these two outcomes:
> t.test(rep(1,5), rep(1,5)) Error in t.test.default(rep(1, 5), rep(1, 5)) : data are essentially constant
I will need a workaround to effectively run a t-test in case 2. I do not have a human available to always look at the data before deciding whether to run the t.test.
I have done some research and saw that some people write exception handling code in these situations. Do I really need to write my own t.test implementation, such as by wrapping exception handling code around the standard stats t.test and then doing some things depending on various things in the data (I don't know what exactly)? If there is a different implementation available somewhere then that might be the best solution. I am not confident in my ability to write new t.test implementations accurately.
These two cases represent entirely possible and realistic experimental data in my situation.
Edit: Additional background seems to be interesting to the community. Here you go.
Thanks to all for so many questions! Let me see if I can provide some of the context the community seems to be wanting. In the experiments in question, we might have a scenario where my users wish to infer whether technique A for studying for SAT exams is better than technique B. IN this case there will be (effectively) continuous outcomes in the form of the mean SAT score by groups A and B. In a different scenario my users want to analyze, there may be an AB test of a web site wherein the user wants to try to get more people to open the email and to do so the firm sends out an A subject on some emails, and a B subject on emails to different group of customers. Whether the customer opened the email is a success, encoded as 1, failure being the recipient did not open the email and this would be encoded most naturally as 0. A comparison of the group means would be interesting, as a significant difference in mean success rate of opening emails could cause the user's firm to change they way they write email subject lines. Hope this helps.