(Pulls Conover  off the bookshelf...)
This idea is quite old; it dates back at least to van der Waerden (1952/1953) , who suggested a test that corresponds to the Kruskal Wallis but with ranks replaced by normal scores. (The idea of using ordered random normal values rather than an approximation of their expectation or their median - is perhaps even a little older.)
According to Conover, Fisher and Yates (1957)  suggest replacing observations with expected normal scores (i.e. transformed ranks) in a variety of tests where normality would be assumed.
The asymptotic relative efficiency at the normal will be 1, which makes it sound quite attractive ... however, the advantage over say the Wilcoxon-Mann-Whitney (gain in power) -- even at the normal -- is quite small, and if the distribution is heavier tailed than normal (say logistic), it may be disadvantageous to do this. (Some simulation suggests that it is in fact the case: unless the distribution is close to normal already -- in which case there's no benefit to doing the transformation -- such a transformation may actually lose power.)
Chernoff & Lehmann  calculate asymptotic power for a variety of distributions; where there's at least one very short tail (such as the uniform), the normal scores test can have much better ARE for a shift alternative against the Wilcoxon-Mann-Whitney -- better than the t-test itself does. Their results agree with my simulations for heavier tailed cases.
Note that in the two-sample case, as the separation in means becomes large, while the combined sample looks quite normal, the two samples are not normal at all:
As a result, not all properties of the normal test will carry over to the normal scores test, and the behaviour at larger separations (with small samples) may be somewhat counterintuitive.
The tests obtained by this idea are sometimes collectively called normal scores tests, which search term (via Google, say) turns up a number of references.
For example, here, Richard Darlington discusses doing it for the Wilcoxon signed ranks test; he points out there's an advantage over the plain rank test, because it reduces the number of tied values of the test statistic.
Before I end up writing pages on it, I'll leave you to search further.
Conover lists a number of other references and has a fair bit of discussion, so I'd definitely recommend reading that.
Gelman's point, however, seems to be about convenience - not needing to develop a new test each time the situation changes; though if convenience is the main issue there's already the ability to use permutation tests on whatever statistic we like. [With the normal scores approach, the difficulty is we still need a suitable way to rank -- you can't just rank things that aren't comparable under the null and expect the right sort of behaviour. There's a similar problem with the permutation test, since you similarly need exchangeability under the null.]
You mention an R function, but you can rank and convert to normal scores easily in R just using functions that already come with R.
e.g. using the
sleep data in R. you'd do a t-test this way:
t.test(extra ~ group, data = sleep) # Welch
t.test(extra ~ group, data = sleep, var.equal=TRUE) # equal-variance
t.test(qqnorm(extra,plot=FALSE)$x ~ group, data = sleep) # normal scores
 Conover, W. J. (1980),
Practical Nonparameteric Statistics, 2e.
Wiley. pp. 316–327.
(From the above Wikipedia link it looks like in 3e (1999) the discussion starts on p396)
 van der Waerden, B.L. (1952),
"Order tests for the two-sample problem and their power",
Proceedings of the Koninklijke Nederlandse Akademie van Wetenschappen, Serie A 55 (Indagationes Mathematicae 14), 453–458.
 van der Waerden, B.L. (1953),
"Order tests for the two-sample problem. II, III",
Proceedings of the Koninklijke Nederlandse Akademie van Wetenschappen, Serie A 56 (Indagationes Mathematicae, 15), 303–310 & 311–316.
(there are also corrections to the 1952 paper on p 80 of that volume)
 Fisher R.A. and Yates F. (1957)
Statistical Tables for Biological, Agricultural and Medical Research, 5e, Oliver & Boyd, Edinburgh.
 Hodges, J. L.; Lehmann, E. L. (1961),
"Comparison of the Normal Scores and Wilcoxon Tests,"
Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, 307--317,
University of California Press, Berkeley, Calif.