# When (and why) should you take the log of a distribution (of numbers)?

Say I have some historical data e.g., past stock prices, airline ticket price fluctuations, past financial data of the company...

Now someone (or some formula) comes along and says "let's take/use the log of the distribution" and here's where I go WHY?

Questions:

1. WHY should one take the log of the distribution in the first place?
2. WHAT does the log of the distribution 'give/simplify' that the original distribution couldn't/didn't?
3. Is the log transformation 'lossless'? I.e., when transforming to log-space and analyzing the data, do the same conclusions hold for the original distribution? How come?
4. And lastly WHEN to take the log of the distribution? Under what conditions does one decide to do this?

I've really wanted to understand log-based distributions (for example lognormal) but I never understood the when/why aspects - i.e., the log of the distribution is a normal distribution, so what? What does that even tell and me and why bother? Hence the question!

UPDATE: As per @whuber's comment I looked at the posts and for some reason I do understand the use of log transforms and their application in linear regression, since you can draw a relation between the independent variable and the log of the dependent variable. However, my question is generic in the sense of analyzing the distribution itself - there is no relation per se that I can conclude to help understand the reason of taking logs to analyze a distribution. I hope I'm making sense :-/

In regression analysis you do have constraints on the type/fit/distribution of the data and you can transform it and define a relation between the independent and (not transformed) dependent variable. But when/why would one do that for a distribution in isolation where constraints of type/fit/distribution are not necessarily applicable in a framework (like regression). I hope the clarification makes things more clear than confusing :)

This question deserves a clear answer as to "WHY and WHEN"

• Because this covers almost the same ground as previous questions here and here, please read those threads and update your question to focus on any aspects of this issue that haven't already been addressed. Note, too, #4 (and part of #3) are elementary questions about logarithms whose answers are readily found in many places.
– whuber
Nov 23, 2011 at 20:46
• The clarification helps. You might want to ponder the fact, though, that regression with only a constant term (and no other independent variables) amounts to assessing the variation of the data around their mean. Therefore, if you really understand the effects of taking logs of dependent variables in regression, you already understand the (simpler) situation you are asking about here. In short, once you have answers to all four questions for regression, you don't need to ask them again about "the distribution in isolation."
– whuber
Nov 23, 2011 at 21:15
• @whuber: I see...so I do understand the reasons for taking logs in regression, but only because I had been taught so - I understand it from the need to do so perspective i.e., to make sure the data fits within the assumptions of linear regression. That's my only understanding. Maybe what I'm missing is "real understanding" of the effect of taking logs and hence the confusion...any help? ;)
– PhD
Nov 23, 2011 at 21:24
• Ah, but you know much more than that, because after using logs in regression, you know that the results are interpreted differently and you know to take care in back-transforming fitted values and confidence intervals. I'm suggesting that you might not be confused and that you probably already know many of the answers to these four questions, even though you weren't initially aware of it :-).
– whuber
Nov 23, 2011 at 21:29
• Readers here may also want to look at these closely related threads: interpretation-of-log-transformed-predictor, & How to interpret logarithmically transformed coefficients in linear regression. Mar 3, 2013 at 2:41

If you assume a model form that is non-linear but can be transformed to a linear model such as $$\log Y = \beta_0 + \beta_1t$$ then one would be justified in taking logarithms of $$Y$$ to meet the specified model form. In general whether or not you have causal series , the only time you would be justified or correct in taking the Log of $$Y$$ is when it can be proven that the Variance of $$Y$$ is proportional to the Expected Value of $$Y^2$$ . I don't remember the original source for the following but it nicely summarizes the role of power transformations. It is important to note that the distributional assumptions are always about the error process not the observed Y, thus it is a definite "no-no" to analyze the original series for an appropriate transformation unless the series is defined by a simple constant.

Unwarranted or incorrect transformations including differences should be studiously avoided as they are often an ill-fashioned /ill-conceived attempt to deal with unidentified anomalies/level shifts/time trends or changes in parameters or changes in error variance. A classic example of this is discussed starting at slide 60 here http://www.autobox.com/cms/index.php/afs-university/intro-to-forecasting/doc_download/53-capabilities-presentation where three pulse anomalies (untreated) led to an unwarranted log transformation by early researchers. Unfortunately some of our current researchers are still making the same mistake.

## Several common used variance-stabilizing transformations

Relationship of $$\sigma^2$$ to $$E(y)$$ Transformation
$$\sigma^2 \propto$$ constant $$y'=y$$ (no transformation)
$$\sigma^2 \propto E(y)$$ $$y' = \sqrt y$$ (square root: Poisson data)
$$\sigma^2 \propto E(y)(1-E(y))$$ $$y' = sin^{-1}(\sqrt y)$$ (arcsin; binomial proportions $$0\le y_i \le 1$$)
$$\sigma^2 \propto (E(y))^2$$ $$y'=log(y)$$
$$\sigma^2 \propto (E(y))^3$$ $$y' = y^{-1/2}$$ (reciprocal square root)
$$\sigma^2 \propto (E(y))^4$$ $$y' = y^{-1}$$ (reciprocal)

The optimal power transformation is found via the Box-Cox Test where

• -1. is a reciprocal
• -.5 is a recriprocal square root
• 0.0 is a log transformation
• .5 is a square toot transform and
• 1.0 is no transform.

Note that when you have no predictor/causal/supporting input series, the model is $$Y_t=u +a_t$$ and that there are no requirements made about the distribution of $$Y$$ BUT are made about $$a_t$$, the error process. In this case the distributional requirements about $$a_t$$ pass directly on to $$Y_t$$. When you have supporting series such as in a regression or in a Autoregressive–moving-average model with exogenous inputs model (ARMAX model) the distributional assumptions are all about $$a_t$$ and have nothing whatsoever to do with the distribution of $$Y_t$$. Thus in the case of ARIMA model or an ARMAX Model one would never assume any transformation on $$Y$$ before finding the optimal Box-Cox transformation which would then suggest the remedy (transformation) for $$Y$$. In earlier times some analysts would transform both $$Y$$ and $$X$$ in a presumptive way just to be able to reflect upon the percent change in $$Y$$ as a result in the percent change in $$X$$ by examining the regression coefficient between $$\log Y$$ and $$\log X$$. In summary, transformations are like drugs some are good and some are bad for you! They should only be used when necessary and then with caution.

• I agree that whomever left the downvote(s) should leave a remark as to why this was downvoted. To Irishstat, it would be much easier to read your post if you took advantage of the formatting options for leaving answers, especially those available for marking up equations in latex. See the markdown editing help section. That link is available whenever you type a response in the top right corner of the posting box (in the orange circle with the question mark). Nov 28, 2011 at 13:22
• The cited table is found in Introduction to Linear Regression Analysis By Douglas C. Montgomery, Elizabeth A. Peck, G. Geoffrey Vining. Jun 19, 2017 at 18:03
• @user1717828 tu .. I have always been a fan of Montgomery as he has a long beard involving time series Jun 19, 2017 at 21:17
• Is it not always true that the second moment and the variance are proportional to one another? We have the classic equation saying: variance is equal to the second moment minus the first moment squared. May 8, 2019 at 13:38
• As you say the variance is a function of the second moment. Where did I imply otherwise. Additionally the variance can change (deterministically) at different points in time SEE pdfs.semanticscholar.org/09c4/… which is not remedied by a power transform. May 8, 2019 at 17:50

Log-scale informs on relative changes (multiplicative), while linear-scale informs on absolute changes (additive). When do you use each? When you care about relative changes, use the log-scale; when you care about absolute changes, use linear-scale. This is true for distributions, but also for any quantity or changes in quantities.

Note, I use the word "care" here very specifically and intentionally. Without a model or a goal, your question cannot be answered; the model or goal defines which scale is important. If you're trying to model something, and the mechanism acts via a relative change, log-scale is critical to capturing the behavior seen in your data. But if the underlying model's mechanism is additive, you'll want to use linear-scale.

Example. Stock market.
Stock A on day 1: $\$$100. On day 2, \$$101. Every stock tracking service in the world reports this change in two ways! (1) +$\$$1. (2) +1%. The first is a measure of absolute, additive change; the second a measure of relative change. Illustration of relative change vs absolute: Relative change is the same, absolute change is different Stock A goes from \$$1 to $\$$1.10. Stock B goes from \$$100 to$\$$110. Stock A gained 10%, stock B gained 10% (relative scale, equal) ...but stock A gained 10 cents, while stock B gained \$$10 (B gained more absolute dollar amount)

If we convert to log space, relative changes appear as absolute changes.

Stock A goes from $\log_{10}(\$1)$to$\log_{10}(\$1.10)$ = 0 to .0413
Stock B goes from $\log_{10}(\$100)$to$\log_{10}(\$110)$ = 2 to 2.0413

Now, taking the absolute difference in log space, we find that both changed by .0413.

Both of these measures of change are important, and which one is important to you depends solely on your model of investing. There are two models. (1) Investing a fixed amount of principal, or (2) investing in a fixed number of shares.

Model 1: Investing with a fixed amount of principal.

Say yesterday stock A cost $\$$1 per share, and stock B costs \$$100 a share. Today they both went up by one dollar to$\$$2 and \$$101 respectively. Their absolute change is identical ($\$$1), but their relative change is dramatically different (100% for A, 1% for B). Given that you have a fixed amount of principal to invest, say \$$100, you can only afford 1 share of B or 100 shares of A. If you invested yesterday you'd have$\$$200 with A, or \$$101 with B. So here you "care" about the relative gains, specifically because you have a finite amount of principal.

Model 2: fixed number of shares.

In a different scenario, suppose your bank only lets you buy in blocks of 100 shares, and you've decided to invest in 100 shares of A or B. In the previous case, whether you buy A or B your gains will be the same ($\$$100 - i.e.$1 for each share).

Now suppose we think of a stock value as a random variable fluctuating over time, and we want to come up with a model that reflects generally how stocks behave. And let's say we want to use this model to maximize profit. We compute a probability distribution whose x-values are in units of 'share price', and y-values in probability of observing a given share price. We do this for stock A, and stock B. If you subscribe to the first scenario, where you have a fixed amount of principal you want to invest, then taking the log of these distributions will be informative. Why? What you care about is the shape of the distribution in relative space. Whether a stock goes from 1 to 10, or 10 to 100 doesn't matter to you, right? Both cases are a 10-fold relative gain. This appears naturally in a log-scale distribution in that unit gains correspond to fold gains directly. For two stocks whose mean value is different but whose relative change is identically distributed (they have the same distribution of daily percent changes), their log distributions will be identical in shape just shifted. Conversely, their linear distributions will not be identical in shape, with the higher valued distribution having a higher variance.

If you were to look at these same distributions in linear, or absolute space, you would think that higher-valued share prices correspond to greater fluctuations. For your investing purposes though, where only relative gains matter, this is not necessarily true.

Example 2. Chemical reactions. Suppose we have two molecules A and B that undergo a reversible reaction.

$A\Leftrightarrow B$

which is defined by the individual rate constants

($k_{ab}$) $A\Rightarrow B$ ($k_{ba}$) $B\Rightarrow A$

Their equilibrium is defined by the relationship:

$K=\frac{k_{ab}}{k_{ba}}=\frac{[A]}{[B]}$

Two points here. (1) This is a multiplicative relationship between the concentrations of $A$ and $B$. (2) This relationship isn't arbitrary, but rather arises directly from the fundamental physical-chemical properties that govern molecules bumping into each other and reacting.

Now suppose we have some distribution of A or B's concentration. The appropriate scale of that distribution is in log-space, because the model of how either concentration changes is defined multiplicatively (the product of A's concentration with the inverse of B's concentration). In some alternate universe where $K^*=k_{ab}-k_{ba}=[A]-[B]$, we might look at this concentration distribution in absolute, linear space.

That said, if you have a model, be it for stock market prediction or chemical kinetics, you can always interconvert 'losslessly' between linear and log space, so long as your range of values is $(0,\inf)$. Whether you choose to look at the linear or log-scale distribution depends on what you're trying to obtain from the data.

EDIT. An interesting parallel that helped me build intuition is the example of arithmetic means vs geometric means. An arithmetic (vanilla) mean computes the average of numbers assuming a hidden model where absolute differences are what matter. Example. The arithmetic mean of 1 and 100 is 50.5. Suppose we're talking about concentrations though, where the chemical relationship between concentrations is multiplicative. Then the average concentration should really be computed on the log scale. This is called the geometric average. The geometric average of 1 and 100 is 10! In terms of relative differences, this makes sense: 10/1 = 10, and 100/10 = 10, ie., the relative change between the average and two values is the same. Additively we find the same thing; 50.5-1= 49.5, and 100-50.5 = 49.5.

• This is a really helpful answer and I love the examples. Could you add more about "when" specifically to use log-transform? You say "When you care about relative changes, use the log-scale; when you care about absolute changes, use linear-scale." But are there cases when you care about relative changes but shouldn't log-transform, and if so, how do you detect those cases? For example, this paper makes a case that data that doesn't follow a log normal distribution shouldn't be log transformed: ncbi.nlm.nih.gov/pmc/articles/PMC4120293 Nov 27, 2017 at 20:19
• @skeller88 I agree with this paper; it is a narrow response to the more broad (and philosophical!) question of 'why do we transform distributions?' I think the answer is that we have a well developed statistical toolkit for contrasting between normal distributions, but a less developed toolkit for other, perhaps even unnamed distributions (the majority). An approach to evaluating a funky looking distribution could be to take the log of it just to see if it looks more normal; but as IrishStat describes technically above, this path is fraught with danger (of the square peg, round hole variety). Nov 28, 2017 at 22:17
• There is a relevant explanation of this effect and why it matters for decision trees a little more towardsdatascience.com/… Dec 31, 2018 at 19:20

I wanted to give an answer in the simplist form. If exponents are short hand for multiplication, and log is the inverse of exponentiation, the taking the log of something is a form of division.

Take the simplest function form y = C. Let C be 100,000 so we have y=100,000. If ws dona log() transform we have y=5.

If we had another function on the same plot of y=1,000,000 it would be hard to graph those together given the range on the y axis. But if we use log() on both now we have functions y=5 and y= 6.

Extend this to simple linear form of y = mx + C and you can see how powerful this can be as things get increasing poweful.

To use a one senetence analogy log transform is equivalent to the scale on a map that says 1in = 1 mile. We dont want a map where 1 mile = 1 mile.. Logarithms scale down when we need it. Exponents scale up. We use both for normalizing data

Why use log?

1.To avoid numerical underflow / overflow

In statistical inference or parameter learning processes, it's very common to cumulate product a series of probability densities. But some times the individual densities are too small (or too big) that computer won't be able to store their product. For example we want to calculate a likelihood $$L=p_1 \cdot p_2$$ where $$p_1=8e^{-300}$$ and $$p_2=6e^{-300}$$, but if you multiply them together in a computer you will get $$L=0$$, because the true result $$4.8e^{-601}$$ is smaller than the smallest positive number a computer can handle. Hence we always use log probabilities or log probability densities during computation.

2.To improve model learning efficiency by exploiting log concave/convex/linear property

We know that parameter learning in essence is an optimazation problem, we also know that if a function is concave/convex/linear, then it's optimal value can be easily found. Most of the common distributions we see are log concave/convex, some are even log linear, which means that the log of the density function is concave/convex/linear, finding it's optimal values in the log space can be much more efficient.

When use log?

As explained in "Why use log?", it is recommended to use log densities/probabilities for all inference and model learning processes.

• Although the question does refer to taking the log "of the distribution," in comments it becomes clear that it means taking the logarithms of the data values.
– whuber
Aug 19, 2020 at 14:07
• @whuber you are right, shame I didn't read the complete question ;) Aug 19, 2020 at 15:35