$R^2$ of Log transformed data is positive, however that of reversed transformed data is negative I am running an XGBoost model with a continuous target variable. With ~200 features I am getting a Test $R^2$ of 0.54. By looking at the distribution of the target variable, it appears it's highly left skewed. So, I took log transformation on the target variable and reran the model which gave me an $R^2$ of 0.68, which is significantly better than the non-transformed model. Then, I reverse transformed the predicted values (antilog) and calculated $R^2$ using original values of the target variable. The $R^2$ is -0.02. 
I am having a difficult time wrapping my head around such results. I understand taking antilog (or exponent) will significantly shoot up the error
$log(y) = f(x) + error$
$y = exp(f(x) + error)$
I am trying to understand which results should I trust and if $R^2$ is the correct accuracy metric to look at?
Case 1: Use the model with original non-transformed data ($R^2 = 0.54$)
Case 2: Use the model with log-transformed target variable ($R^2 = 0.68$, but when reverse transformed $R^2 = -0.02$)
 A: I used to deal with a similar problem few years ago. When you inverse the log-treansform, 0.2 unit difference in log scale may be 1000 in real (inversed) value.
The best practice to maximize your metric is to develop a model using the transformed variable BUT calculate the metric on real (inversed) variable.
As a result, you may need to use a customized scoring function if you use GridSearchCV() of sklearn package.
To make a scorer, you can use something like following code:
def r2new(y_true, y_pred):
    return r2_score(np.exp(y_true), np.exp(y_pred))

r2new_scorer = make_scorer(r2new, greater_is_better=True)

xgb_clf.GridSearchCV(..., scoring=r2new_scorer)

y_log=np.log(y)
xgb_clf.fit(X,y_log)
y_log_calc=xgb_clf.predict(X)
print("R2: %s"%r2_scorer(y, np.exp(y_log_calc)))

A: In general Case 2 is the best option. 
Considering that $R^2$ is dependent from the sum of squared residuals, it is highly affected by high values in skewed distributions. 
Assuming your dataset has five points with $y = [10, 12, 15, 13, 105]$, then two models with predictions $\hat{y}=[5,7,10,8,105]$ and $\hat{y}=[10,12,15,13,115]$ would have the same $R^2$ with a sum of squared residuals of $100$, even though the errors of the first are much bigger in relative terms! (of course this is an exagerated example with an outlier, but works the same for skewed distributions). 
When you transform to the log, values become more comparable and you reduce the relative importance of tail observations! 

Finally, it all boils down to what your data represents and what type of error you want to minimize. If missing a prediction by 10 points is the same to you both for a true value of 100 and 10000, then use non-transformed variables. If (like in most cases) you care more about making close prediction on the bulk of the data rather than the tail, then go with the log transformation.
