Statistical power of binomial vs. paired t-test I am conducting an experiment and I am deciding whether to use a forced-choice paradigm (participants choose A or B), or a continuous rating paradigm (rate each stimulus A and B on a Likert scale).
There are a lots of issues to consider, but I am specifically interested in how this choice affects my statistical power to detect an effect.
I know, for instance, that a binomial test could not detect a difference at p<.05 with fewer than six subjects, whereas it's possible that a paired t-test could.
Practically speaking, does a paired t-test usually have higher power than a binomial test (or for that matter, chi-square?)
 A: The simple answer is that all else being equal, if you have a scale and dichotomize it into just A or B, you will generally have less power than if you used the continuous scale (see e.g. Senn, Stephen. "Disappointing dichotomies." Pharmaceutical Statistics 2.4 (2003): 239-240. or Senn, Stephen. "Being efficient about efficacy estimation." Statistics in Biopharmaceutical Research 5.3 (2013): 204-210.). You should of course consider whether somehow all else is not equal. E.g. it could be possible that one particular way of asking a question (e.g. forcing people to make a clear choice) is somehow better at elicitating a clear preference. However, I would like to see some pretty compelling evidence for that before I would abandon a more nuanced outcome in favour of a dichotomized outcome.
A: Besides the issue if dichotomous vs continuous variables as discussed by @Bjorn, the is the issue of paired vs non-paired tests. The dichotomous correspondent of the paired t-test is not a binomial test (which is unpaired) but the McNemar test - a paired dichotomous test. In my experience, paired tests are more powerful than non-paired tests, but I don't know of any proof of that.
