Pitfalls to avoid when transforming data? I achieved a strong linear relationship between my $X$ and $Y$ variable after doubly transforming the response. The model was 
$Y\sim X$
 but I transformed it to 
$\sqrt{\frac{Y}{X}}\sim \sqrt{X}$
improving $R^2$ from .19 to .76. 
Clearly I did some decent surgery on this relationship. Can anyone discuss the pitfalls of doing this, such as dangers of excessive transformations or possible violations of statistical principles?
 A: There are two elements to @Peter's example, which it might be useful to disentangle:
(1) Model mis-specification. The models
$$y_i = \beta_0 + \beta_1 x_i + \varepsilon_i \qquad\text{(1)}$$
&
$$w_i=\gamma_0 + \gamma_1 z_i + \zeta_i \qquad\text{(2)}$$
, where $w_i=\sqrt{\frac{y_i}{x_i}}$ & $z_i=\sqrt{x_i}$, can't both be true. If you re-express each in terms of the other's response they become non-linear in the parameters, with heteroskedastic errors. 
$$w_i = \sqrt{\frac{\beta_0}{z_i^2} + \beta_1 + \frac{\varepsilon_i}{z_i^2}} \qquad\text{(1)}$$
$$y_i = (\gamma_0 \sqrt x_i + \gamma_1 \sqrt x_i + \zeta_i \sqrt x_i)^2 \qquad\text{(2)}$$
If $Y$ is assumed to be a Gaussian random variable independent of $X$, then that's a special case of Model 1 in which $\beta_1=0$, & you shouldn't be using Model 2. But equally if $W$ is assumed to be a Gaussian random variable independent of $Z$, you shouldn't be using Model 1. Any preference for one model rather than the other has to come from substantive theory or their fit to data.
(2) Transformation of the response. If you knew $Y$ & $X$ to be independent Gaussian random variables, why should the relation between $W$ & $Z$ still surprise you, or would you call it spurious? The conditional expectation of $W$ can be approximated with the delta method:
$$ \operatorname{E} \sqrt\frac{Y}{x} = \frac{\operatorname{E}\sqrt{Y}}{z} \\
 \approx \frac{\sqrt{\beta_0} + \frac{\operatorname{Var}{Y}}{8\beta_0^{3/2}}}{z}$$
It is indeed a function of $z$.
Following through the example ...
set.seed(123)
x <- rnorm(100, 20, 2)
y <- rnorm(100, 20, 2)
w <- (y/x)^.5
z <- x^.5
wrong.model <- lm(w~z)
right.model <- lm(y~x)
x.vals <- as.data.frame(seq(15,25,by=.1))
names(x.vals) <- "x"
z.vals <- as.data.frame(x.vals^.5)
names(z.vals) <- "z"
plot(x,y)
lines(x.vals$x, predict(right.model, newdata=x.vals), lty=3)
lines(x.vals$x, (predict(wrong.model, newdata=z.vals)*z.vals)^2, lty=2)
abline(h=20)
legend("topright",legend=c("data","y on x fits","w on z fits", "truth"), lty=c(NA,3,2,1), pch=c(1,NA,NA,NA))
plot(z,w)
lines(z.vals$z,sqrt(predict(right.model, newdata=x.vals))/as.matrix(z.vals), lty=3)
lines(z.vals$z,predict(wrong.model, newdata=z.vals), lty=2)
lines(z.vals$z,(sqrt(20) + 2/(8*20^(3/2)))/z.vals$z)
legend("topright",legend=c("data","y on x fits","w on z fits","truth"),lty=c(NA,3,2,1), pch=c(1,NA,NA,NA))



Neither Model 1 nor Model 2 is much use for predicting $y$ from $x$, but both are all right for predicting $w$ from $z$: mis-specification hasn't done much harm here (which isn't to say it never will—when it does, it ought to be apparent from the model diagnostics). Model-2-ers will run into trouble sooner as they extrapolate further away from the data—par for the course, if your model's wrong. Some will gain pleasure from contemplation of the little stars they get to put next to their p-values, while some Model-1-ers will bitterly grudge them this—the sum total of human happiness stays about the same. And of course, Model-2-ers, looking at the plot of $w$ against $z$, might be tempted to think that intervening to increase $z$ will reduce $w$—we can only hope & pray they don't succumb to a temptation we've all been incessantly warned against; that of confusing correlation with causation.
Aldrich (2005), "Correlations Genuine and Spurious in Pearson and Yule", Statistical Science, 10, 4 provides an interesting historical perspective on these issues.
A: The earlier answer of @Glen_b is all important.  Playing with transformations distorts every part of statistical inference and results in $R^2$ that is biased high.  In short, not having a parameter in the model for everything you don't know will give a false sense of precision.  That's why regression splines are now so popular.
A: You can't really compare $R^2$ before and after, because the underlying variability in $Y$ is different. So you literally can take no comfort whatever from the change in $R^2$. That tells you nothing of value in comparing the two models.
The two models are different in several ways, so they mean different things -- they assume very different things about the shape of the relationship and the variability of the error term (when considered in terms of the relationship between $Y$ and $X$). So if you're interested in modelling $Y$ (if $Y$ itself is meaningful), produce a good model for that. If you're interested in modelling $\sqrt Y$ (/$\sqrt Y$ is meaningful), produce a good model for that. If $\sqrt{Y/X}$ carries meaning, then make a good model for that. But compare any competing models on comparable scales. $R^2$ on different responses simply aren't comparable.
If you're just trying different relationships in the hope of finding a transformation with a high $R^2$ -- or any other measure of 'good fit' -- the properties of any inference you might like to conduct will be impacted by the existence of that search process.
Estimates will tend to be biased away from zero, standard errors will be too small, p-values will be too small, confidence intervals too narrow. Your models will on average appear to be 'too good' (in the sense that their out-of-sample behavior will be disappointing compared to in-sample behavior).
To avoid this kind of overfitting, you need, if possible, to do the model-identification and estimation on different subsets of the data (and model evaluation on a third). If you repeat this kind of procedure on many "splits" of the data taken at random, you get a better sense of how reproducible your results are.
There are many posts here with relevant points on these issues: it might be worth trying some searches.
(If you have good a priori reasons for choosing a particular transformation, that's a different issue. But searching the space of transformations to find something that fits carries all manner of 'data snooping' type problems with it.)
A: There's a bigger problem than the ones identified by @Glen_b . 
set.seed(123)
x <- rnorm(100, 20, 2)
y <- rnorm(100, 20, 2)
dv <- (y/x)^.5
iv <- x^.5
m1 <- lm(dv~iv)
summary(m1)

And I get an $R^2$ of 0.49 and a P-value that is $5.5 \times 10^{-16}$.
You have $X$ on both sides of the equation.
