1
$\begingroup$

I want to perform a linear regression analysis. The distributions of data for all continuous variables are not normal. The tail of graph is to the right and thre highest point of graph is due to the left. (I know that in this situation we can change data by using log. (if the highest point is in the right and the tail of graph in the left ^2 could be used.) Now I think Log 10 is a better option. What about Ln or log 2? Is there a rule about the type of Log in this situation?

$\endgroup$
20
  • 3
    $\begingroup$ Do you prefer inches or centimeters? Pounds or kilograms? That's the only difference among any two bases of logarithm: a constant multiplicative factor. $\endgroup$ – whuber Oct 17 '19 at 22:22
  • 1
    $\begingroup$ 1. None of the variables are assumed to be normal for linear regression analysis. If you're doing testing or confidence intervals or prediction intervals based off the most commonly used approaches, the error term is assumed to be normal in those (corresponding to the conditional distribution of the response). You can't assess that by looking at the marginal distribution of the response variable (nor any of the predictors). 2. It makes no difference to anything which base of logs you use when you do take logs (as long as you're consistent with your choice). $\endgroup$ – Glen_b Oct 17 '19 at 22:23
  • 1
    $\begingroup$ 1. Unfortunately, you can't believe everything you read on random sites on the internet. The person who wrote that is simply wrong; it's easy to see what the assumptions are if you know what things you use in deriving the results for tests and intervals (which they clearly have never done). I suggest you avoid visiting it again -- if they're that clueless about what the assumptions are in regression, who knows what else they might say. ... ctd $\endgroup$ – Glen_b Oct 18 '19 at 0:24
  • 1
    $\begingroup$ No, I am sorry, I must protest. This has been well understood since the normal-based hypothesis tests (F- and t-tests), CI's & PIs for regression were first derived. There has been no controversy on it at any time in about 90(?) years nor any recent change that could excuse the magnitude of that error. There have, however, been a large number of people who get it wrong, simply because they've never even looked at the actual mathematics involved but relied on statements from other people who also have never looked at it. They share their ignorance widely and it gets passed down generations. $\endgroup$ – Glen_b Oct 18 '19 at 1:50
  • 1
    $\begingroup$ @Slow Jason W. Osborne's PhD is in ... educational psychology. I will happily defer to him on that topic. His main reference for the claim of multivariate normality as an assumption appears to be Tabachnik & Fidell, which is brim full of errors of exactly this kind. Note that both of them are psychologists as well. So this is precisely the problem I was referring to. A person whose actual training in psychology, using other psychologists to support their claims in a publication outside of statistics does not in any sense constitute a controversy in statistics. $\endgroup$ – Glen_b Oct 18 '19 at 2:38
1
$\begingroup$

If you are solving a plain OLS analytically (i.e. using psuedoinverse, as in most statistical packages), then the base of the log doesn't matter since it will just be a constant multiplicative factor like @whuber mentioned.

But if you have a large dataset and solving the regression via some iterative algorithm such as stochastic gradient descent, then choosing a proper log base could help with convergence speed.

  • Pick a log base such that the variance of the transformed attribute is comparable to that of other attributes
  • Picking the wrong log base will still converge to the same answer eventually, it just affects convergence speed
$\endgroup$
3
  • $\begingroup$ I have a hard time believing this: could you demonstrate how simply rescaling the response values by the relatively smalll amounts involved in converting between bases $2,$ $e,$ and $10$ could have any effect on convergence rates? Note, too, that most (good) software performs an internal initial standardization of the data, which means that by the time it begins iterating, the units of measurement don't matter at all. $\endgroup$ – whuber Oct 18 '19 at 13:07
  • $\begingroup$ I don't think the typical internal standardization handles lognormal transformation, but do let me know if there's software/packages that does that and I'd love to know more $\endgroup$ – aranelladen Oct 18 '19 at 18:11
  • $\begingroup$ In terms of scaling/normalizing with a different base, it depends on the algorithm but the general idea is that a feature with much higher variance than the others could dominate the next descent step, thus slowing down convergence. You can read more here: en.wikipedia.org/wiki/Feature_scaling $\endgroup$ – aranelladen Oct 18 '19 at 18:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy