5
$\begingroup$

lately I lost access to SPSS and instead of using Python or R, I tend to perform analysis using a free software called Jamovi.

The thing is, this software doesn't have the different non-linear regression models I liked to use in SPSS, so I tend to make only linear-regression models.

Because of this I became used to perform log transforms in the data prior to making the model. I noticed that most of the time, after transforming a predictor in the linear regression model, the R squared increased.

I know that log transform can reduce the skewness of the data, but I wonder why it always seems to improve the linear-regression model (only using the log transformed variable as a predictor, not the original variable).

Is this a true improvement or is it some kind of statistical artifact? Why does this happen? Should I use it every time?

$\endgroup$
3

2 Answers 2

12
$\begingroup$

There's no reason why this would always improve your $R^2$, most likely it is because you are forcing a linear fit where it isn't ideal and your transformation 'linearizes' the predictor somewhat.

Here's a quick counterexample where the data generating mechanism is in fact linear:

set.seed(1)
x <- runif(1e2)
y <- x + rnorm(1e2, 0, 0.1)

summary(fit <- lm(y ~ x))$r.squared           # 0.897
summary(lfit <- lm(y ~ log(x)))$r.squared     # 0.736

The first 100,000 seeds all show a better $R^2$ for the untransformed predictor.

$\endgroup$
7
$\begingroup$

It doesn't. Here is a very simple example where $R^2$ is virtually identical (but a TINY bit higher before the transform). This is R code, anything after a # is a comment

set.seed(1234)

x <- rnorm(1000, 10, 1)
y <- 3*x + rnorm(1000, 0, 5)

m1 <- lm(y~x)
m2 <- lm(y~log(x))

summary(m1) #0.3083
summary(m2) #0.3027

and, if I run the code again (without the first line, so, a different seed) the untransformed version has $R^2$ of 0.30 and the transformed one, 0.27.

$\endgroup$
5
  • 1
    $\begingroup$ Oh, thank you, I guess the data I use tends to benefit from that transform whenever I use it coincidentally. $\endgroup$
    – gabriel
    Oct 17, 2023 at 12:42
  • 1
    $\begingroup$ You're welcome. Since you are new here, I'll say that, if an answer meets your needs, it is common to "accept" it by clicking the check mark. $\endgroup$
    – Peter Flom
    Oct 17, 2023 at 12:46
  • 2
    $\begingroup$ But it still begs the question if most data tends to benefit from the transformation anyways. If I used it blindly, it might be that most of the time the transformation leads to an improvement in R2. Specially if I deal with skewed data. $\endgroup$
    – gabriel
    Oct 17, 2023 at 12:46
  • 3
    $\begingroup$ @gabriel It's not necessarily a benefit, it's just different. I wouldn't base a transformation decision on differences in $R^2$. $\endgroup$
    – mkt
    Oct 17, 2023 at 12:54
  • 3
    $\begingroup$ "Most data" is too vague to be evaluated. But it's certainly possible that, if your data are very right skew, this could help. Also, if your variable is skewed your errors are often skewed, and this violates an assumption. $\endgroup$
    – Peter Flom
    Oct 17, 2023 at 12:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.