I'm aware there's a great number of questions which deal with the mathematical difference between the two, but I'm still confused as to best practice.
Basically I'm looking at a situation where we have lots of bad data, and some good data (which we can assume accurately measures the variable we're concerned with). The current approach is to regress the good data onto the bad data for what is basically a training set, i.e. $$good = m*bad+c+\epsilon$$
This is what I mean by reverse regression. We then use the ample bad data to predict what the good data would have been if it were collected elsewhere (think of this as the testing stage).
When I first saw this, I was convinced it was wrong. The bad data has the measurement errors, and the 'correct' approach is to fit a model regressing the bad data onto the good data, then invert the model to predict in the opposite direction. This is what I mean by inverse regression.
However I've seen since been playing with some toy models and I'm starting to see that the 'wrong' method produces much better predictions.
n <- 1000 x <- runif(n,-10,10) m <- 0.7 c <- 4 e <- rnorm(n,0,3) y <- m*x+c+e df <- data.frame(x=x,y=y) traindf <- df[1:(4*n/5),] testdf <- df[(4*n/5+1):n,] mod1 <- lm(y ~ x, data=traindf) mod2 <- lm(x ~ y, data=traindf) preds1 <- (testdf$y-coef(mod1))/coef(mod1) preds2 <- predict(mod2, newdata=testdf) mean((preds1-testdf$x)^2)  18.62054 mean((preds2-testdf$x)^2)  12.50204
Now I get why this is the case, model 2 is designed to reduce mean square error when predicting x from y, it's hardly surprising it does a better job at doing it. And I also understand the geometrical interpretations, one model is minimising vertical distances between the points and the line-of-best fit, whereas the other is minimising the horizontal differences etc. I get the maths.
What I don't understand is, why should I care that the second model is misspecified if it's giving me better predictions regardless?
Are there any advantages to using the 'correct' model? Are there other situations where inverse regression actually outperforms? Maybe there are other loss functions it does well with, or it's more robust to violations of assumptions, or more wrong but less biased. But so far it looks like reverse regression produces better point estimates and I'm going to bootstrap my confidence intervals in any case. It's also less of a hassle than manually inverting your model or playing around with the library investr.
What am I missing? This isn't an academic question as the data produced matters somewhat.