In Rosenthal and Rubin (1979) ``A Note on Percent Variance Explained as A Measure of the Importance of Effects'', they give an example of where $r^2$ is deceptively low:
Suppose half the patients in a medical study are randomly assigned to a new medical treatment (X = 1) while the other half are assigned the standard medical treatment (X = 0). The dependent variable is “alive one year after treatment” (Y = 1) vs. “dead” (Y = 0). Suppose the study obtained the frequencies displayed in Table 1. Under the standard medical treatment, 30%of the patients live; while under the new treatment, 70% of the patients live. This certainly is a dramatic and important effect. Yet, for these data r2 = .16. The conclusion that the treatment is unimportant because it accounts for only 16%of the variance is simply wrong. Percent variance explainied can, in some cases, then, be a very deceptive measure.
The numbers are
| Dead (Y=0) | Alive (Y=1) |
-------------------------------------
X = 0 | 35 | 15 | 50
X = 1 | 15 | 35 | 50
-------------------------------------
| 50 | 50 | 100
How do you actually calculate the $r^2$ here?