Theory question: How to use Mean Absolute Error properly in a log scaled linear regression

Question

First of all, I had a look here and in a couple of other questions: I couldn't find what I am looking for.

So my question is purely theoretical (although I have an example by my hands).

Suppose I have some data $(x_i,y_i)$ for $i=1,..,n$. Suppose I fit the following models with IID $\epsilon_i \sim N(0, \sigma^2)$ for $i=1,..,n$

• $M_1: \log(y_i)= \beta_0+\beta_1x_i+\epsilon_i$
• $M_2: \log(y_i)= \beta_0+\beta_1x_i+\beta_2x_i^2+\epsilon_i$
• $M_3: \log(y_i)= \beta_0+\beta_1x_i+\beta_2x_i^2+\beta_3x_i^3+\epsilon_i$

Now I want to see which of these models is better, so I use the following (maybe weird, but stay with me) method, to evaluate their "predictive powers":

1. Use $(x_i, \log(y_i))$ for $i=1,..,\frac{n}{2}$, to fit $M_1, M_2, M_3$ respectively.
2. Now use the fitted model (so $M_1, M_2,M_3$ respectively), to predict $y_i$'s using the $x_i$'s from the remaining $\frac{n}{2}$ data , so from $i = \frac{n}{2}+1, .., n$ (careful, predict $y_i$ not $\log(y_i)$)
3. Use MAE or Mean Absolute Error (here) $MAE = \frac{1}{\frac{n}{2}}\sum_{i=\frac{n}{2}+1}^{n}|y_i-\hat{y}_i|$, being careful that $\hat{y}_i$ is in the original scale of values!

So now my question:

If I do point $1.$ and I fit the three models (hence obtaining estimates for the parameters, their standard errors etc..) and then use these parameters (respectively of course!) to predict the responses of the other $x_i$'s:

1. Will I be predicting $\log(y_i)$'s right? And this is true... Is it also true that in order to get $\hat{y}_i$'s , instead of $\widehat{\log{(y)}}_i$, I should just take the exponential of those terms? So in general, is it true $\hat{y}_i = e^{\widehat{\log{(y)}}_i}$?
2. Once I find the three MAE's, how do I judge the models? Should I be looking for the one with smaller MAE?

Answer 1

For 1, the answer here really depends on what you want to predict. The value $\hat{y}_i$ is the thing you want to predict. I'll elucidate with an example: If you want to predict the mean value of the $y_i$s then what you've proposed with $\hat{y}_i = e^{\widehat{log(y_i)}}$ will be an under estimate of what you want (because of the convexity of the map $x \mapsto e^{x}$). However, because the exponential map is monotonic, the map preserves quantiles and using your normality distribution error assumption the prediction $e^{\widehat{log(y_i)}}$ will be a prediction of the median of $y_i | x_i$. To estimate the mean of $y_i \vert x_i$ you'll need an estimate of $\sigma$ because the mean of $y_i$ in your model will be $e^{\widehat{log(y_i)} +\hat{\sigma}^2/2}$. This follows from the Moment Generating Function (MGF) of a normal distribution and setting $t=1$ (the variable of the MGF).

For 2, if this is the criteria you've decided on then choosing the model amongst those 3 that gives the smallest value of the loss function you choose would be a common choice. If you do go the exponential route and predict the median then choosing a sum of absolute deviations would be the logically paired loss function based on that estimator.

One thing to keep in mind with the three models propose is that there is likely a substantial degree of colinearity because the covariates are polynomial functions of one another, e.g. $x_i$, $x_i^2$, and $x_i^3$.