Comparing a Log10 Transformed Multiple Linear Regression Model MSE to untransformed MSE

Question

I have a base MLR model of data that has 8 numeric predictors and 3 categorical predictors. The response is whole number numeric values.

My base model has some mild regression assumption violations (non-normality of residuals shown below) so I tamed them by utilizing a log10 transform.

I transformed the response as well as the numeric variables and left the categorical predictors alone. I also offset each predictor by 1 + min(that variable) to get around any sort of NAs that are results of the log10 transform (for negative values).

Now my goal is to compare the MSE from this model to other future models (Random Forest, SVM, Regularized Regression, etc.). I want to be able to compare this model's MSE to other models (that don't use transformations). So I need to get the residuals in terms of original units.

How exactly would I go about doing so? The inverse of y = log10(x) is of course 10^y = x but how can I do this with the model's $residuals to get the residuals in non-transformed units? Since some of my variables are categorical how does this effect my un-transformation?

# R Code for MLR model
mlr_fit_3_log <- lm(log10(Rented_Bike_Count + 1 - min(bike2$Rented_Bike_Count)) ~ 
                    log10(Hour + 1 min(bike2$Hour)) +
                    log10(Temperature + 1 - min(bike2$Temperature)) + 
                log10(Humidity + 1 - min(bike2$Humidity)) +
                    log10(Wind_speed + 1 - min(bike2$Wind_speed)) +
                log10(Visibility + 1 - min(bike2$Visibility)) +
                    log10(Solar_Radiation + 1 - min(bike2$Solar_Radiation))+
                log10(Rainfall + 1 - min(bike2$Rainfall)) + 
                    log10(Snowfall + 1 - min(bike2$Snowfall)) +
                    Seasons + # factor
                    Holiday + # factor
                    Functioning_Day # factor 
                    - Dew_point_temperature, data = Train_set)

# Calculate MSE  model
mse <- function(fit_obj){
  mean(fit_obj$residuals^2)
}

Answer 1

You can't really make that comparison in a useful way, as the results of the two models are on different scales. This might be easier to think about with a continuous outcome variable rather than a count variable like you have. Say your outcome variable was instead the number of dollars spent on renting bikes, taking into account both the number rented and the cost per hour of rental.

If you use the number of dollars as the outcome variable, the square root of the MSE will also be in dollars. Logarithms, in contrast, are unitless. When you take the log of the number of dollars as an outcome variable, you are really modeling the log of the ratio of the number of dollars to a single base unit of 1 dollar. That's explained nicely here in the context of distances. As a result, the square root of the mean square error represents something related to fractional errors in your predictions rather than absolute errors. You could perhaps choose some illustrative representative examples, but there is no unique relationship between the two measures of MSE that would be very useful to a reader.

In terms of your particular application, instead of jumping through the hoops you devised to formulate your model (taking differences between actual and minimum values of counts, adding 1 to avoid taking the log of 0, etc), you probably would be better off modeling the counts directly with a generalized linear model, like a Poisson regression with a log link function. Also, a log transform of the outcome variable (or a generalized model with a log link) doesn't necessarily require logarithmic transformations of the predictors. Transformation of predictors should be done to ensure that the net linear predictor (sum of predictors weighted by their coefficients) is linearly related to the (generalized) outcome. Whether log transforms of predictors helps with that depends on the situation.