0
$\begingroup$

I have been running a log-transform on my target values because the distribution appears to be highly right skewed as you can see in the picture.enter image description here

After having called

df['target'] = np.log(df['target'])

the distribution target looks like this enter image description here

that is way better than before for training a model.

At this point I run the ML process and I train my model on log-scaled targets getting the following predictions (still using the log-scaled targets):

enter image description here

obtained by simply plotting log scaled predictions against log scaled true values, where the red line is the 'ideal' linear relationship between predictions and targets that I'm trying to achieve.

I got an R2 score of 0.40 which is not amazing but is not too bad at the moment.

The problem is, that when I try to get back to the original values by an inverse transform, i.e.

preds = np.exp(model.predict(X_test))
y_test = np.exp(y_test)

then I get the following:

enter image description here

and a R2 score of -0.090 obtained by running

r2_score(y_test, preds)

(hence using the inverse transformed values).

What am I doing wrong?

thank you in advance,

James

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.