# Using Box-Cox transformed features as input decreased the $R^2$ score of a regression model

I am working on building a regression model to predict housing sales price using house features (Ames housing dataset). And I prepared feature set in two ways

Case 1.

I performed boxcox transformation on all the numerical features and performed one hot encoding on the categorical features.

Case 2.

I used the numerical feature set as it is and performed one hot encoding on the categorical features.

I applied linear regression on both feature sets and got r2 score much much lower in Case 1 (where numerical features were transformed). In Case 2 it was always more than 0.85 for any train test combination and in Case 1 it was never more than 0.5.

Since linear regression assumes the features to be conditionally independent and normal distributed and box cox helps to make features near normally distributed I thought performing box cox will boost my r2 score. Can anyone help me understand why did the r2 score plummeted for the model applied on box-cox transformed data?

• Your understanding of linear regression, unfortunately, is quite wrong. There is no assumption that features should be independent or normal. Maybe the untransformed variant contains strong outliers that can trigger leverage effects, which can affect the R-squared. Working with a clean train/valid/test strategy could reveal what is going on. Jun 5 at 21:03
• If you transformed the response variable, then the $R^2$ values just aren't comparable. Not even remotely. That's because they compare reductions in the variances of two different sets of data.
– whuber
Jun 6 at 0:33
• @MichaelM Thank you for the insight. I did further research and discovered that the normality of the features is not an assumption. In fact, I should have figured that while performing one hot encoding (that it will obviously not result in a perfect or near normally distributed dummy variables all the time). However does not independence of the features is one of the assumptions in a ideal case linear regression? That is what I make of the lack of multi-collinearity assumption from this article: en.wikipedia.org/wiki/Linear_regression#Assumptions Jun 7 at 5:47
• @whuber Appreciate your input! By response variable do you mean the dependent variable or output variable (or the Y in Y = B*X). In my case the dependent variable was in consistent format in both the cases. Jun 7 at 5:52
• @whuber, yes in both cases the response variable used in model was log of actual response variable. But I get your point. Jun 8 at 4:27

Consequently, when you transform the features, there is no reason to expect that being normal gets you closer to some ideal that should result in higher performance. You aren’t assured of worse performance, either, but if you transform features that are roughly linearly related to $$y$$, you might wind up with a relationship between the transformed feature and $$y$$ that is not linear, and your linear model will miss such a relationship.