Am I looking for a better behaved distribution for the independent variable in question, or to reduce the effect of outliers, or something else?
I always hesitate to jump into a thread with as many excellent responses as this, but it strikes me that few of the answers provide any reason to prefer the logarithm to some other transformation that "squashes" the data, such as a root or reciprocal.
Before getting to that, let's recapitulate the wisdom in the existing answers in a more general way. Some non-linear re-expression of the dependent variable is indicated when any of the following apply:
(These indications can conflict with one another; in such cases, judgment is needed.)
So, when is a logarithm specifically indicated instead of some other transformation?
Finally, some non - reasons to use a re-expression:
I always tell students there are three reasons to transform a variable by taking the natural logarithm. The reason for logging the variable will determine whether you want to log the independent variable(s), dependent or both. To be clear throughout I'm talking about taking the natural logarithm.
Firstly, to improve model fit as other posters have noted. For instance if your residuals aren't normally distributed then taking the logarithm of a skewed variable may improve the fit by altering the scale and making the variable more "normally" distributed. For instance, earnings is truncated at zero and often exhibits positive skew. If the variable has negative skew you could firstly invert the variable before taking the logarithm. I'm thinking here particularly of Likert scales that are inputted as continuous variables. While this usually applies to the dependent variable you occasionally have problems with the residuals (e.g. heteroscedasticity) caused by an independent variable which can be sometimes corrected by taking the logarithm of that variable. For example when running a model that explained lecturer evaluations on a set of lecturer and class covariates the variable "class size" (i.e. the number of students in the lecture) had outliers which induced heteroscedasticity because the variance in the lecturer evaluations was smaller in larger cohorts than smaller cohorts. Logging the student variable would help, although in this example either calculating Robust Standard Errors or using Weighted Least Squares may make interpretation easier.
The second reason for logging one or more variables in the model is for interpretation. I call this convenience reason. If you log both your dependent (Y) and independent (X) variable(s) your regression coefficients ($\beta$) will be elasticities and interpretation would go as follows: a 1% increase in X would lead to a ceteris paribus $\beta$% increase in Y (on average). Logging only one side of the regression "equation" would lead to alternative interpretations as outlined below:
Y and X -- a one unit increase in X would lead to a $\beta$ increase/decrease in Y
Log Y and Log X -- a 1% increase in X would lead to a $\beta$% increase/decrease in Y
Log Y and X -- a one unit increase in X would lead to a $\beta*100$ % increase/decrease in Y
Y and Log X -- a 1% increase in X would lead to a $\beta/100$ increase/decrease in Y
And finally there could be a theoretical reason for doing so. For example some models that we would like to estimate are multiplicative and therefore nonlinear. Taking logarithms allows these models to be estimated by linear regression. Good examples of this include the Cobb-Douglas production function in economics and the Mincer Equation in education. The Cobb-Douglas production function explains how inputs are converted into outputs:
$$Y = A L^\alpha K^\beta $$
$Y$ is the total production or output of some entity e.g. firm, farm, etc.
$A$ is the total factor productivity (the change in output not caused by the inputs e.g. by technology change or weather)
$L$ is the labour input
$K$ is the capital input
$\alpha$ & $\beta$ are output elasticities.
Taking logarithms of this makes the function easy to estimate using OLS linear regression as such:
$$\log(Y) = A + \alpha\log(L) + \beta\log(K)$$
One typically takes the log of an input variable to scale it and change the distribution (e.g. to make it normally distributed). It cannot be done blindly however; you need to be careful when making any scaling to ensure that the results are still interpretable.
This is discussed in most introductory statistics texts. You can also read Andrew Gelman's paper on "Scaling regression inputs by dividing by two standard deviations" for a discussion on this. He also has a very nice discussion on this at the beginning of "Data Analysis Using Regression and Multilevel/Hierarchical Models".
Taking the log is not an appropriate method for dealing with bad data/outliers.
For more on whuber's excellent point about reasons to prefer the logarithm to some other transformations such as a root or reciprocal, but focussing on the unique interpretability of the regression coefficients resulting from log-transformation compared to other transformations, see:
Oliver N. Keene. The log transformation is special. Statistics in Medicine 1995; 14(8):811-819. DOI:10.1002/sim.4780140810. (PDF of dubious legality available at http://rds.epi-ucsf.org/ticr/syllabus/courses/25/2009/04/21/Lecture/readings/log.pdf).
If you log the independent variable x to base b, you can interpret the regression coefficient (and CI) as the change in the dependent variable y per b-fold increase in x. (Logs to base 2 are therefore often useful as they correspond to the change in y per doubling in x, or logs to base 10 if x varies over many orders of magnitude, which is rarer). Other transformations, such as square root, have no such simple interpretation.
If you log the dependent variable y (not the original question but one which several of the previous answers have addressed), then I find Stephen Cole's idea of 'sympercents' attractive for presenting the results (i even used them in a paper once), though they don't seem to have caught on all that widely:
Tim J Cole. Sympercents: symmetric percentage differences on the 100 log(e) scale simplify the presentation of log transformed data. Statistics in Medicine 2000; 19(22):3109-3125. DOI:10.1002/1097-0258(20001130)19:22<3109::AID-SIM558>3.0.CO;2-F [I'm so glad Stat Med stopped using SICIs as DOIs...]
You tend to take logs of the data when there is a problem with the residuals. For example, if you plot the residuals against a particular covariate and observe an increasing/decreasing pattern (a funnel shape), then a transformation may be appropriate. Non-random residuals usually indicate that your model assumptions are wrong, i.e. non-normal data.
Some data types automatically lend themselves to logarithmic transformations. For example, I usually take logs when dealing with concentrations or age.
Although transformations aren't primarily used to deal outliers, they do help since taking logs squashes your data.
|show 2 more comments|
Shane's point that taking the log to deal with bad data is well taken. As is Colin's regarding the importance of normal residuals. In practice I find that usually you can get normal residuals if the input and output variables are also relatively normal. In practice this means eyeballing the distribution of the transformed and untransformed datasets and assuring oneself that they have become more normal and/or conducting tests of normality (e.g. Shapiro-Wilk or Kolmogorov-Smirnov tests) and determining whether the outcome is more normal. Interpretablity and tradition are also important. For example, in cognitive psychology log transforms of reaction time are often used, however, to me at least, the interpretation of a log RT is unclear. Furthermore, one should be cautious using log transformed values as the shift in scale can change a main effect into an interaction and vice versa.