18
$\begingroup$

I'm studying statistics and often come across formulae containing the log and I'm always confused if I should interpret that as the standard meaning of log, i.e. base 10, or if in statistics the symbol log is generally assumed to be the natural log ln.

In particular I'm studying the Good-Turing Frequency Estimation as an example, but my question is more of general one.

$\endgroup$
  • 2
    $\begingroup$ "For many applications, the natural logarithm of the likelihood function, called the log-likelihood, is more convenient to work with." en.wikipedia.org/wiki/Likelihood_function#Log-likelihood In statistics we often work with likelihood function, it is usually the ln that is considered. However, the two are related: log(x) = ln(x) / ln(10) = ln(x) / 2.303, and the ln-likelihood function reaches the extremum at the same point as log10-likelihood function. $\endgroup$ – John_West Apr 3 '16 at 22:30
  • 5
    $\begingroup$ In a few particular application areas, when $\log$ is mentioned, base 10 is intended, but as Aksakal indicates, otherwise it's the convention used in mathematics -- that an unadorned $\log$ means natural log. $\endgroup$ – Glen_b -Reinstate Monica Apr 3 '16 at 23:53
  • 2
    $\begingroup$ As @John_West says $ln(x)$ and $log_a(x)$ are identical up to a scaling factor. So they are the same only that you measure in another unit. $\endgroup$ – user83346 Apr 4 '16 at 7:30
  • 1
    $\begingroup$ @Aksakal; what you say comes to saying that the unit is important (see my comment supra), which I agree with. I also wrote $log_a$ to explicitly indicate the base. For (some) applications in statistics like maximimum likelihood, this scaling factor is however irrelavant. The maximum will not change after adding the scaling factor. In the OP's reference (good-turing...) they want to plot $log(N_r)$ (or $log(Z_r)$) versus $log(r)$. This means that the unit changes on both axes of the plot so the plotted ''curve'' does not change. $\endgroup$ – user83346 Apr 4 '16 at 13:57
  • 1
    $\begingroup$ Unless you're writing a paper, even when using log-likelihood the scale (base of logarithm) usually matters. For instance, the log likelihood ratio test statistics uses $\ln$, you'd have to adjust from other base to use the critical values. If you're writing software, it's important to get the base right when using log likelihood functions from papers etc. There's just too many cases where base is important to state that it doesn't matter. $\endgroup$ – Aksakal Apr 4 '16 at 14:30
20
$\begingroup$

It's safe to assume that without explicit base $\log=\ln$ in statistics, because base 10 log is not used very often in statistics. However, other posters bring up a point that $\log_{10}$ or other bases can be common in some other fields, where statistics is applied, e.g. information theory. So, when you read papers in other fields, it gets confusing at times.

Wikipedia's entropy page is a good example of confusing usage of $\log$. In the same page they mean base 2, $e$ and any base. You can figure out by the context which one is meant, but it requires reading the text. This is not a good way to present the material. Compare it to Logarithm page where the base is clearly shown in every formula or $\ln$ is used. I personally think this is the way to go: always show the base when $\log$ sign is used. This would also be ISO compliant for the standard doesn't define usage of unspecified base with $\log$ symbol as @Henry pointed out.

Finally, ISO 31-11 standard prescribes $\text{lb}$ and $\lg$ signs for base 2 and 10 logarithms. Both are rarely used these days. I remember that we used $\lg$ in high school, but that was in another century in another world. I have never seen it since used in a statistical context. There isn't even the tag for $\text{lb}$ in LaTeX.

$\endgroup$
  • 1
    $\begingroup$ Base 2 logarithms are also quite common in some fields. Unadorned log is rarely base 10, but it's not always base e. $\endgroup$ – Nuclear Wang Apr 4 '16 at 9:03
  • $\begingroup$ Helpful, but I think "rarely" is too strong. There are substantive fields in which people may know only about, or at best feel most familiar with, base 10 logarithms. Note that many graphs show logarithmic scales using powers of 10. Someone preferring natural logarithms finds no difficulty decoding such scales, but the presumption is of base 10. $\endgroup$ – Nick Cox Apr 4 '16 at 14:03
  • $\begingroup$ @NickCox, OP specifically states "statistics" as a field, and I don't see base 10 logarithm used in statistics often. $\endgroup$ – Aksakal Apr 4 '16 at 14:05
  • $\begingroup$ ISO 31-11 seems to specify $\ln$ for $\log_e$, and leave an unadorned $\log$ undefined $\endgroup$ – Henry Apr 4 '16 at 14:10
  • 1
    $\begingroup$ @NickCox, I softened the language, you bring up a fair point $\endgroup$ – Aksakal Apr 4 '16 at 14:22
14
$\begingroup$

It depends.

Outside of a few contexts, like converting a value to decibels, base 10 logarithms are pretty rare in equations. However, log-scale plots are often in base-10, though this should be pretty easy to verify from the labels on the axes.

In a mathematical context, an unadorned $\log$ is likely to be the natural log (i.e., $\log_{e}$ or $\ln$). On the other hand, computer science often uses base-2 logarithms ($\log_2$), and they're not always clearly marked as such. The good news is that you can convert between bases trivially and using the "wrong" base will only make your answer off by a constant factor.

In Gale's 1995 "Good-Turing Without Tears" paper, the logarithms in the text actually are $\log_{10}$ (it says so on page 5), but the R/S+ code in the appendix uses the log function, which is actually $\log_e$ or $\ln$. As @Henry points out below, this makes no practical difference.

If I were forced to guess, here are some heuristics:

  • If powers of 2, $e$, or 10 are also present, the logs are likely to have the corresponding base.

  • If it arises from integrating $1/x$ (or, more generally, involves calculus), it's likely to be a natural log.

  • If it arises from repeatedly dividing something in half (as in binary search), it's likely to be $\log_2$. More generally, something can be divided by $n$ approximately $\log_n$ times.

  • Information-theoretic calculations typically use $\log_2$, especially in modern work. However, you can check the units to be sure: $\textrm{bits} \rightarrow \log_2$, $\textrm{nats} \rightarrow \ln$, and $\textrm{bans} \rightarrow \log_{10}$.

  • Finding the point where a function falls or rises to $\frac{1}{e} \textrm{ or } 1- \frac{1}{e}$, (37% and 63%, respectively) of an initial value suggests a natural log.

$\endgroup$
  • 5
    $\begingroup$ +1. A small tip is that if exponentials $\exp()$ are found nearby then natural logarithm is more likely and conversely with powers of 10 or 2. If which base is being used remains unclear, try to reproduce authors' example calculations. $\endgroup$ – Nick Cox Apr 4 '16 at 8:25
  • 2
    $\begingroup$ Since the graphs on pages 6 and 7 of Gale's paper show the original units on a log scale, and the calculations are aimed at the slope of a log-log relationship, i.e. $b$ in the expression $\log(N_r ) = a + b \log(r)$ which corresponds to $N_r=Ar^b$, it makes no practical difference in this case $\endgroup$ – Henry Apr 4 '16 at 9:13
  • 2
    $\begingroup$ Another example of $base 10$ is when platting stock market data, when using a log price axis it is always base 10. $\endgroup$ – Marcus D Apr 4 '16 at 14:07
3
$\begingroup$

To answer your question: no, you cannot assume a general fixed notation for the logarithm.

A similar question was recently discussed in SE.Math: What is the difference between the three types of logarithms? from a mathematical point of view. Generally, there are different notations that depend on habits ($\log_{10}$ seems of use in medical research) or language (for instance in German, Russian, French). Unfortunately, the same notation sometimes ends up representing different definitions. Quoting from the above SE.Math link:

Notation $\ln x$ (almost) unambiguously denotes the natural logarithm $\log_e x$ (latin: logarithmus naturalis), or logarithm in base $e$. The notation $\log x$ should be the adopted notation for the natural logarithm, and it is so in mathematics. However, it often represents the "most natural" depending on the field: I learned it as the base-$*10$ logarithm ($\log_{10}$) at school, and it is often used this way in engineering (for instance in the definition of decibels)

Quite often, if you are not concerned with the meaning of physical units (like decibels @Matt Krause), nor interested in specific rates of change (in biostatistics, the $\log$-ratio for fold-change often denotes the base-$2$ logarithm $\log_2$), it is likely that the natural logarithm ($\log_e$) is used.

For instance, in power or Box-Cox transforms (for variance stabilization), the natural logarithm appears as a limit when the exponent tends to $0$.

Going back to your initial motivation, the Good-Turing Frequency Estimation, it is interesting to read The Population Frequencies of Species and the Estimation of Population Parameters, I. J. Good, Biometrika, 1953. Here, he used logarithmms in different contexts: variable transformation for variance stabilisation (mentioning Bartlett and Anscombe), sum of harmonic series, entropy. We see that he generally uses $\log$ as the natural logarithm, and once in a while in the paper specifies $\log_e$ or $\log_{10}$, when the context requires it. For variance stabilization, or basic entropy estimation, a factor on the logarithm does not change much the result, as the outcome allows a linear change.

$\endgroup$
0
$\begingroup$

In the Akaike Information Criterion the base is $e$, and $\ln(\hat L)$ of the maximum likelihood $\hat L$ is being compared additively to the number of parameters $k$: $$ \mathrm{AIC} = 2(k-\ln(L)).$$

Thus it seems that if you use any other base for the logarithm in the AIC, you may end up drawing the wrong conclusion and selecting the wrong model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.