I have a general question regarding proportions. Let's say, for simplification of things, that I have 2 groups of subjects (treatment and control), and for each subject I measure success or failure (0 or 1). I wish to check if there is a significant difference between the groups. Assuming for now that there are no covariates, nor random effects, how do I choose the appropriate test ? I have a test for proportion difference (with the Z statistic for large sample sizes), I have other tests for proportion difference (I guess, like there are many CI's). On the other hand, I can use contingency tables, Chi Square Test, or Fisher Exact Test. A third option, the odds ratio and the relative risk...How does one chooses the appropriate test, or approach, how do I know if to go for proportion difference, or Fisher Exact Test, or odds ratios ?
This problem may be ideally suited for Chi-square test or Fisher exact test for determination of significance of the difference (P value). Odds ratio can be added in the same analysis as a measure of 'effect size' to show how much better or worse is treatment as compared to control. One can also determine the confidence interval of the odds ratio (or relative risk) to show the degree of spread. Hence, analysis and reporting of multiple tests will clarify the data much more than any single value and these are commonly done in medical articles.
Edit: Relative risk vs odds ratio has been extensively discussed. From https://en.wikipedia.org/wiki/Relative_risk#Comparison_to_the_odds_ratio :
In epidemiological research, the odds ratio is commonly used for case-control studies, as odds, but not probabilities, are usually estimated. Relative risk is used in randomized controlled trials and cohort studies.