When (and why) should you take the log of a distribution (of numbers)?

Question

Say I have some historical data e.g., past stock prices, airline ticket price fluctuations, past financial data of the company...

Now someone (or some formula) comes along and says "let's take/use the log of the distribution" and here's where I go WHY?

Questions:

1. WHY should one take the log of the distribution in the first place?
2. WHAT does the log of the distribution 'give/simplify' that the original distribution couldn't/didn't?
3. Is the log transformation 'lossless'? I.e., when transforming to log-space and analyzing the data, do the same conclusions hold for the original distribution? How come?
4. And lastly WHEN to take the log of the distribution? Under what conditions does one decide to do this?

I've really wanted to understand log-based distributions (for example lognormal) but I never understood the when/why aspects - i.e., the log of the distribution is a normal distribution, so what? What does that even tell and me and why bother? Hence the question!

UPDATE: As per @whuber's comment I looked at the posts and for some reason I do understand the use of log transforms and their application in linear regression, since you can draw a relation between the independent variable and the log of the dependent variable. However, my question is generic in the sense of analyzing the distribution itself - there is no relation per se that I can conclude to help understand the reason of taking logs to analyze a distribution. I hope I'm making sense :-/

In regression analysis you do have constraints on the type/fit/distribution of the data and you can transform it and define a relation between the independent and (not transformed) dependent variable. But when/why would one do that for a distribution in isolation where constraints of type/fit/distribution are not necessarily applicable in a framework (like regression). I hope the clarification makes things more clear than confusing :)

This question deserves a clear answer as to "WHY and WHEN"

Answer 1

If you assume a model form that is non-linear but can be transformed to a linear model such as $\log Y = \beta_0 + \beta_1t$ then one would be justified in taking logarithms of $Y$ to meet the specified model form. In general whether or not you have causal series , the only time you would be justified or correct in taking the Log of $Y$ is when it can be proven that the Variance of $Y$ is proportional to the Expected Value of $Y^2$ . I don't remember the original source for the following but it nicely summarizes the role of power transformations. It is important to note that the distributional assumptions are always about the error process not the observed Y, thus it is a definite "no-no" to analyze the original series for an appropriate transformation unless the series is defined by a simple constant.

Unwarranted or incorrect transformations including differences should be studiously avoided as they are often an ill-fashioned /ill-conceived attempt to deal with unidentified anomalies/level shifts/time trends or changes in parameters or changes in error variance. A classic example of this is discussed starting at slide 60 here http://www.autobox.com/cms/index.php/afs-university/intro-to-forecasting/doc_download/53-capabilities-presentation where three pulse anomalies (untreated) led to an unwarranted log transformation by early researchers. Unfortunately some of our current researchers are still making the same mistake.

Several common used variance-stabilizing transformations

Relationship of $\sigma^2$ to $E(y)$	Transformation
$\sigma^2 \propto$ constant	$y'=y$ (no transformation)
$\sigma^2 \propto E(y)$	$y' = \sqrt y$ (square root: Poisson data)
$\sigma^2 \propto E(y)(1-E(y))$	$y' = sin^{-1}(\sqrt y)$ (arcsin; binomial proportions $0\le y_i \le 1$)
$\sigma^2 \propto (E(y))^2$	$y'=log(y)$
$\sigma^2 \propto (E(y))^3$	$y' = y^{-1/2}$ (reciprocal square root)
$\sigma^2 \propto (E(y))^4$	$y' = y^{-1}$ (reciprocal)

The optimal power transformation is found via the Box-Cox Test where

• -1. is a reciprocal
• -.5 is a recriprocal square root
• 0.0 is a log transformation
• .5 is a square toot transform and
• 1.0 is no transform.

Note that when you have no predictor/causal/supporting input series, the model is $Y_t=u +a_t$ and that there are no requirements made about the distribution of $Y$ BUT are made about $a_t$, the error process. In this case the distributional requirements about $a_t$ pass directly on to $Y_t$. When you have supporting series such as in a regression or in a Autoregressive–moving-average model with exogenous inputs model (ARMAX model) the distributional assumptions are all about $a_t$ and have nothing whatsoever to do with the distribution of $Y_t$. Thus in the case of ARIMA model or an ARMAX Model one would never assume any transformation on $Y$ before finding the optimal Box-Cox transformation which would then suggest the remedy (transformation) for $Y$. In earlier times some analysts would transform both $Y$ and $X$ in a presumptive way just to be able to reflect upon the percent change in $Y$ as a result in the percent change in $X$ by examining the regression coefficient between $\log Y$ and $\log X$. In summary, transformations are like drugs some are good and some are bad for you! They should only be used when necessary and then with caution.

Answer 2

Log-scale informs on relative changes (multiplicative), while linear-scale informs on absolute changes (additive). When do you use each? When you care about relative changes, use the log-scale; when you care about absolute changes, use linear-scale. This is true for distributions, but also for any quantity or changes in quantities.

Note, I use the word "care" here very specifically and intentionally. Without a model or a goal, your question cannot be answered; the model or goal defines which scale is important. If you're trying to model something, and the mechanism acts via a relative change, log-scale is critical to capturing the behavior seen in your data. But if the underlying model's mechanism is additive, you'll want to use linear-scale.

Example. Stock market.
Stock A on day 1: $\$$100. On day 2, $\$$101. Every stock tracking service in the world reports this change in two ways! (1) +$\$$1. (2) +1%. The first is a measure of absolute, additive change; the second a measure of relative change.

Illustration of relative change vs absolute: Relative change is the same, absolute change is different
Stock A goes from $\$$1 to $\$$1.10. Stock B goes from $\$$100 to $\$$110.

Stock A gained 10%, stock B gained 10% (relative scale, equal)
...but stock A gained 10 cents, while stock B gained $\$$10 (B gained more absolute dollar amount)

If we convert to log space, relative changes appear as absolute changes.

Stock A goes from $\log_{10}(\$1)$ to $\log_{10}(\$1.10)$ = 0 to .0413
Stock B goes from $\log_{10}(\$100)$ to $\log_{10}(\$110)$ = 2 to 2.0413

Now, taking the absolute difference in log space, we find that both changed by .0413.

Both of these measures of change are important, and which one is important to you depends solely on your model of investing. There are two models. (1) Investing a fixed amount of principal, or (2) investing in a fixed number of shares.

Model 1: Investing with a fixed amount of principal.

Say yesterday stock A cost $\$$1 per share, and stock B costs $\$$100 a share. Today they both went up by one dollar to $\$$2 and $\$$101 respectively. Their absolute change is identical ($\$$1), but their relative change is dramatically different (100% for A, 1% for B). Given that you have a fixed amount of principal to invest, say $\$$100, you can only afford 1 share of B or 100 shares of A. If you invested yesterday you'd have $\$$200 with A, or $\$$101 with B. So here you "care" about the relative gains, specifically because you have a finite amount of principal.

Model 2: fixed number of shares.

In a different scenario, suppose your bank only lets you buy in blocks of 100 shares, and you've decided to invest in 100 shares of A or B. In the previous case, whether you buy A or B your gains will be the same ($\$$100 - i.e. $1 for each share).

Now suppose we think of a stock value as a random variable fluctuating over time, and we want to come up with a model that reflects generally how stocks behave. And let's say we want to use this model to maximize profit. We compute a probability distribution whose x-values are in units of 'share price', and y-values in probability of observing a given share price. We do this for stock A, and stock B. If you subscribe to the first scenario, where you have a fixed amount of principal you want to invest, then taking the log of these distributions will be informative. Why? What you care about is the shape of the distribution in relative space. Whether a stock goes from 1 to 10, or 10 to 100 doesn't matter to you, right? Both cases are a 10-fold relative gain. This appears naturally in a log-scale distribution in that unit gains correspond to fold gains directly. For two stocks whose mean value is different but whose relative change is identically distributed (they have the same distribution of daily percent changes), their log distributions will be identical in shape just shifted. Conversely, their linear distributions will not be identical in shape, with the higher valued distribution having a higher variance.

If you were to look at these same distributions in linear, or absolute space, you would think that higher-valued share prices correspond to greater fluctuations. For your investing purposes though, where only relative gains matter, this is not necessarily true.

Example 2. Chemical reactions. Suppose we have two molecules A and B that undergo a reversible reaction.

$A\Leftrightarrow B$

which is defined by the individual rate constants

($k_{ab}$) $A\Rightarrow B$ ($k_{ba}$) $B\Rightarrow A$

Their equilibrium is defined by the relationship:

$K=\frac{k_{ab}}{k_{ba}}=\frac{[A]}{[B]}$

Two points here. (1) This is a multiplicative relationship between the concentrations of $A$ and $B$. (2) This relationship isn't arbitrary, but rather arises directly from the fundamental physical-chemical properties that govern molecules bumping into each other and reacting.

Now suppose we have some distribution of A or B's concentration. The appropriate scale of that distribution is in log-space, because the model of how either concentration changes is defined multiplicatively (the product of A's concentration with the inverse of B's concentration). In some alternate universe where $K^*=k_{ab}-k_{ba}=[A]-[B]$, we might look at this concentration distribution in absolute, linear space.

That said, if you have a model, be it for stock market prediction or chemical kinetics, you can always interconvert 'losslessly' between linear and log space, so long as your range of values is $(0,\inf)$. Whether you choose to look at the linear or log-scale distribution depends on what you're trying to obtain from the data.

EDIT. An interesting parallel that helped me build intuition is the example of arithmetic means vs geometric means. An arithmetic (vanilla) mean computes the average of numbers assuming a hidden model where absolute differences are what matter. Example. The arithmetic mean of 1 and 100 is 50.5. Suppose we're talking about concentrations though, where the chemical relationship between concentrations is multiplicative. Then the average concentration should really be computed on the log scale. This is called the geometric average. The geometric average of 1 and 100 is 10! In terms of relative differences, this makes sense: 10/1 = 10, and 100/10 = 10, ie., the relative change between the average and two values is the same. Additively we find the same thing; 50.5-1= 49.5, and 100-50.5 = 49.5.

Answer 3

I wanted to give an answer in the simplist form. If exponents are short hand for multiplication, and log is the inverse of exponentiation, the taking the log of something is a form of division.

Take the simplest function form y = C. Let C be 100,000 so we have y=100,000. If ws dona log() transform we have y=5.

If we had another function on the same plot of y=1,000,000 it would be hard to graph those together given the range on the y axis. But if we use log() on both now we have functions y=5 and y= 6.

Extend this to simple linear form of y = mx + C and you can see how powerful this can be as things get increasing poweful.

To use a one senetence analogy log transform is equivalent to the scale on a map that says 1in = 1 mile. We dont want a map where 1 mile = 1 mile.. Logarithms scale down when we need it. Exponents scale up. We use both for normalizing data

Answer 4

A practical answer:

Why use log?

1.To avoid numerical underflow / overflow

In statistical inference or parameter learning processes, it's very common to cumulate product a series of probability densities. But some times the individual densities are too small (or too big) that computer won't be able to store their product. For example we want to calculate a likelihood $L=p_1 \cdot p_2$ where $p_1=8e^{-300}$ and $p_2=6e^{-300}$, but if you multiply them together in a computer you will get $L=0$, because the true result $4.8e^{-601}$ is smaller than the smallest positive number a computer can handle. Hence we always use log probabilities or log probability densities during computation.

2.To improve model learning efficiency by exploiting log concave/convex/linear property

We know that parameter learning in essence is an optimazation problem, we also know that if a function is concave/convex/linear, then it's optimal value can be easily found. Most of the common distributions we see are log concave/convex, some are even log linear, which means that the log of the density function is concave/convex/linear, finding it's optimal values in the log space can be much more efficient.

When use log?

As explained in "Why use log?", it is recommended to use log densities/probabilities for all inference and model learning processes.