Suppose that I am interested in selecting from one of the following model spaces:
$$\begin{align*} y &= \beta_0 + \beta_1 x_i + \epsilon_i \, \, \text{or} \\ \log (y_i) &= \gamma_0 + \gamma_1 x_i + \eta_i. \end{align*}$$
I intend to use cross validation for making my selection.
If I calculate the prediction errors on the validation set, the first model will give errors in $y$, while the second will give errors in $\log(y_i)$. Obviously, it doesn't make sense to compare these.
Instead, for the second model, suppose that I estimate
$$\begin{equation*} \text{E}[y_i \mid x_i] = \exp \left\{\hat{\beta}_0 + \hat{\beta}_1 x_i \right\} \times \exp \left\{ \hat{\sigma}^2 /2 \right\}, \end{equation*}$$
using an assumption that $\epsilon_i \mid x_i \sim N(0, \sigma^2)$ (alternatively, I could just use a Poisson model with robust standard errors/oversidsperion to get the same result; see here for a nice explanation). Then, I compare this predicted $y$ to the observed validation $y$ to get my prediction errors.
Is this procedure sensible? Will it generally have good properties?
