Let me preface this by saying I'm new to statistics.

I'm working with regression models, attempting to understand transformations a bit more. I'm modeling (Y~X) and I get an $R^2$ of 0.4. I see that the residuals of this plot are left skewed so I take (Y^2~X) assuming that would correct the issue but now my $R^2$ is 0.3. Just out of curiosity, I did (Log(Y)~X and got an $R^2$ of 0.5.

I'm really not sure what is going on and not sure what transformation I should use going forward.

  • 2
    $\begingroup$ $R^2$ on models for $Y$ and $t(Y)$ for a nonlinear transformation $t$ are not comparable. See (for example) the discussion here. You also can't compare $s^2$, $AIC$, $BIC$, ... Also see comments here $\endgroup$ – Glen_b Aug 4 '16 at 1:36
  • $\begingroup$ Do plot the data again and again, for your own sake to see what is going on, and to allow us to give specific advice. In each case it is an easy scatter plot and a fitted line. $\endgroup$ – Nick Cox Aug 4 '16 at 8:09
  • 1
    $\begingroup$ It's most unlikely that $Y^2$ ~ $X$ and $\log Y$ ~ $X$ are both serious models for the data. $\endgroup$ – Nick Cox Aug 4 '16 at 12:04

The total sum of squares $\text{SST}=\sum(y_i-\bar y)^2$ will be altered by transformation.

The total variation available to be explained in the three cases ($Y_0=\log Y, Y_1=Y, Y_2=Y^2$) will be different.

Specifically, if $Y$ tends to be substantially larger than $1$, you'll compress the variation by logging it and similarly expand the variation by squaring it (if $Y$ is positive but tends to be much smaller than $1$ then the log transform will stretch it and the square will compress it).

That stretching/compression may tend to explain the changes in your $R^2$.

| cite | improve this answer | |
  • $\begingroup$ This is a bit brief by our standards at the moment. Do you think you could expand on it a little? When you write "the 3 will change", this seems quite ambiguous to me - what is it that the "3" is referring to? $\endgroup$ – Silverfish Aug 4 '16 at 2:13
  • $\begingroup$ @Silverfish Sorry ya I just started using this site so I should have made this a comment. Can I change it to go there instead? $\endgroup$ – VCG Aug 4 '16 at 2:14
  • $\begingroup$ I've flagged it for you so a moderator can convert it. There is a "flag" button next to the "share" and "edit" buttons - at least for me. $\endgroup$ – Silverfish Aug 4 '16 at 2:19
  • $\begingroup$ @Silverfish Thanks and sorry. I'll be more considerate next time. $\endgroup$ – VCG Aug 4 '16 at 2:35
  • 3
    $\begingroup$ Actually, it's so close to a reasonable answer I'd rather edit it than move it.I hope that's okay with you VCG. Feel free to edit. If you really would prefer I roll back to your original brief one and make it a comment, that can still be done $\endgroup$ – Glen_b Aug 4 '16 at 3:07

From the reference I give below: $R^2$ is explained as,

$R^2$ = $Explained \ Variation / Total \ Variation$


1) $R^2$ is always between 0 and 100%:

2) 0% indicates that the model explains none of the variability of the response data around its mean.

3) 100% indicates that the model explains all the variability of the response data around its mean. the "variation divided by the total variation."

Also from another reference:"...The coefficient of determination, $R^2$, is useful because it gives the proportion of the variance (fluctuation) of one variable that is predictable from the other variable. It is a measure that allows us to determine how certain one can be in making predictions from a certain model/graph." In http://mathbits.com/MathBits/TISection/Statistics2/correlation.htm

Since expansion and contraction was covered by the other answer, I just want to make some comments on of aspects that effect the $R^2$. An important part of $R^2$, is the selection of the function used to fit data. You could have functions that would have have the same expansion or contraction and they can have different $R^2$. With the results you mention and the $R^2$ given I would be incline to try a polynomial (in you independent variable) to see how it fits. This function that contains multiple terms that are powers of x that would fit best, which would mean you can try a polynomial fit. In the most general case of the polynomial, you would use spline regression to find an fit. The $R^2$ review is the first step in analyzing data. Note: "Pearson Product-Moment Correlation" (which can be found on the Internet) discusses using $R^2$ to determine the "strength of the correlation."

The general reference on regression (and also "over fitting") I mention above a few times on how to interpret the correlation coefficient is "Regression Analysis: How Do I Interpret R-squared and Assess the Goodness-of-Fit?" is http://blog.minitab.com/blog/adventures-in-statistics/regression-analysis-how-do-i-interpret-r-squared-and-assess-the-goodness-of-fit.

Lastly, just want to make a general comment on function selection vs model. A model is development from the principles and laws of the field of study you are working in. In many case a model is developed even before data is collected (e.g. Theoretical Physics -- I have done this many times). On the other hand just selecting functions to try to "fit" data from experiments is not classified as a model. You are just looking for the best fit to the data (again as a first step) -- then others could study your data and develop/derive a model.

| cite | improve this answer | |
  • 2
    $\begingroup$ It is never valid to use $R^2$ to compare models when you are transforming the regressor variable $x$. Although there are principled methods to search for and identify transformations of $y$, $x$, or both that improve a model, the recommendations in this answer are not among them. $\endgroup$ – whuber Aug 4 '16 at 18:21
  • $\begingroup$ The correlation coefficient can be used to compare models, because it corresponds to the "best fit" for a model. You just don't ever "over fit" when taking this approach. Here is an example where is was used to get the best model: math.usask.ca/~miket/S344D..pdf $\endgroup$ – jimmeh Aug 4 '16 at 23:14
  • 3
    $\begingroup$ That comment is simply wrong, as is made abundantly clear in hundreds--perhaps thousands--of posts here on model fitting, model comparison, and overfitting. Your reference is a generic textbook account of various methods of model selection with OLS (some of them now outmoded or deprecated), without any evaluation of their properties. There's nothing in there that supports your assertions. $\endgroup$ – whuber Aug 4 '16 at 23:16
  • 4
    $\begingroup$ I have to agree with @whuber. The fact that correlation measures linearity in some space e.g. $(x, y)$ doesn't mean that it works well when used to select models by comparing results in some other space e.g. $\log x, \log y$. One of many examples is that a high correlation in one space could be an artefact of an outlier; taking logs could reduce the correlation but on any other grounds produce a configuration better suited to modelling. $\endgroup$ – Nick Cox Aug 5 '16 at 16:19
  • 1
    $\begingroup$ Discussion should be reserved for statistical argument, not personal comment. You can expect any comments making personal references to be removed, even ones that might also contain useful argument or references. $\endgroup$ – Glen_b Aug 5 '16 at 23:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.