67

For $x_2$ and $x_1$ close to each other, the percent change $\frac{x_2-x_1}{x_1}$ approximates the log difference $\log x_2 - \log x_1$. Why does the percent change approximate the log difference? An idea from calculus is that you can approximate a smooth function with a line. The linear approximation is simply the first two terms of a Taylor Series. The ...


48

The log of $1$ is just $0$ and the limit as $x$ approaches $0$ (from the positive side) of $\log x$ is $-\infty$. So the range of values for log probabilities is $(-\infty, 0]$. The real advantage is in the arithmetic. Log probabilities are not as easy to understand as probabilities (for most people), but every time you multiply together two probabilities (...


30

Be careful, because the notation $\mathrm{Ln}$ is currently used in mathematics. For $z\in\mathbb{C}$, the complex logarithm is the multivalued function defined by $$ \mathrm{Ln}(z) = \ln(|z|)+i(\arg(z)+2k\pi) $$ for $k=0,\pm 1,\pm 2, \dots$. Hence, you should check the mentioned papers to verify in which context they use the $\mathrm{Ln}$ notation. Take ...


27

You just need to take exponential of both sides of the equation and you will get a potential relation, that may make sense for some data. $$\log(Y) = a\log(X) + b$$ $$\exp(\log(Y)) = \exp(a \log(X) + b)$$ $$Y = e^b\cdot X^a$$ And since $e^b$ is just a parameter that can take any positive value, this model is equivalent to: $$Y=c \cdot X^a$$ It should ...


25

If we consider "approximation" in a fairly general sense we can get somewhere. We have to assume not that we have an actual normal distribution but something that's approximately normal except the density cannot be nonzero in a neighborhood of 0. So let's say that $a$ is "approximately normal" (and concentrated near the mean*) in a sense that we can ...


25

I fear you have misunderstood what the article intends. This is no great surprise, since it's somewhat unclearly written. There are two different things going on. The first is simply to work on the log scale. That is, instead of "$p_{AB} = p_A\cdot p_B$" (when you have independence), one can instead write "$\log(p_{AB}) = \log(p_A)+ \log(p_B)$". If you ...


25

It's safe to assume that without explicit base $\log=\ln$ in statistics, because base 10 log is not used very often in statistics. However, other posters bring up a point that $\log_{10}$ or other bases can be common in some other fields, where statistics is applied, e.g. information theory. So, when you read papers in other fields, it gets confusing at ...


24

The iris data set is a fine example to learn PCA. That said, the first four columns describing length and width of sepals and petals are not an example of strongly skewed data. Therefore log-transforming the data does not change the results much, since the resulting rotation of the principal components is quite unchanged by log-transformation. In other ...


19

No; sometimes it will make it worse. Heteroskedasticity where the spread is close to proportional to the conditional mean will tend to be improved by taking log(y), but if it's not increasing with the mean at close to that rate (or more), then the heteroskedasticity will often be made worse by that transformation. Because taking logs "pulls in" more ...


19

This is a consequence of Jensen's Inequality. You want $E[y|x]$, but exponentiating the predicted value(s) from the log model will not provide unbiased estimates of $E[y|x]$, as $$E[y_i|x_i] = \exp(x'\beta) \cdot E[\exp(u_i)]$$ and the second term is omitted in your calculation. If the error term $u \sim N[0,\sigma^2]$, then $E[\exp(u)] = \exp(\frac{1}{2}\...


18

I would like to add that taking the log of a probability or probability density can often simplify certain computations, such as calculating the gradient of the density given some of its parameters. This is in particular when the density belongs to the exponential family, which often contain fewer special function calls after being logged than before. This ...


17

If you say your model is ln(y) = b*ln(x) + a it is only part of your model. Your actual model includes an error term: $\ln y_i = b\cdot \ln x_i + a + \varepsilon_i$ and you assume that the error distribution is $\varepsilon_i \sim \mathcal{N}(0,\,\sigma^2)$. Now let's back-transform it: $y_i = \exp(a) \cdot x_i^b \cdot \exp(\varepsilon_i)$ As you see, ...


16

The function $f(x)=\log_{10}(x)$ is the inverse of exponentiation with base 10. It is a monotonic injective function mapping positive numbers to $\mathbb{R}$. Positive numbers less than 1 are mapped to negative numbers. Positive numbers greater than one are mapped to positive numbers. In regression analysis, logarithm transformations are used when effects ...


16

No. For example, if $X$ follows a log normal distribution, where $\log(X) \sim N(\mu,\sigma)$, then $E[\log(x)] = \mu$ and is independent of $\sigma$. However, its mean is $E[X] = \exp \left(\mu + \frac{\sigma^2}{2} \right)$. Clearly, you cannot derive a $\sigma$ dependent number from a $\sigma$ independent number.


15

Since logarithm is only defined for positive numbers, you can't take the logarithm of negative values. However, if you are aiming at obtaining a better distribution for your data, you can apply the following transformation. Suppose you have skewed negative data: x <- rlnorm(n = 1e2, meanlog = 0, sdlog = 1) x <- x - 5 plot(density(x)) then you can ...


15

It depends. Outside of a few contexts, like converting a value to decibels, base 10 logarithms are pretty rare in equations. However, log-scale plots are often in base-10, though this should be pretty easy to verify from the labels on the axes. In a mathematical context, an unadorned $\log$ is likely to be the natural log (i.e., $\log_{e}$ or $\ln$). On ...


14

I've come to the conclusion that you're probably trying to reproduce what Excel does. This is not necessarily sensible, but at least it's pretty straightforward. Here's what Excel does: linear trendline: ordinary simple regression, fitted by least squares logarithmic trendline: $y \sim a + b \ln(x)$ -- fitted by taking $x'=\ln(x)$ & using ordinary ...


14

Logarithmic loss = Logistic loss = log loss = $-y_i\log(p_i) - (1 -y_i) \log(1 -p_i)$ Sometimes people take a different logarithmic base, but it typically doesn't matter. I hear logistic loss more often. FYI: How is logistic loss and cross-entropy related? Thesaurus for statistics and machine learning terms When is log-loss metric appropriate for ...


14

There are two meta-principles here. If in doubt possibly ambiguous notation should be explained. That can be done concisely and just once. Explaining your notation is good scientific and mathematical manners and essential if you want to be easily understood (we won't discuss the opposite case). There are common-sense limits to this. Depending on context ...


14

You can take your model $\log(Y)=a\log(X)+b$ and calculate the total differential, you will end up with something like : $$\frac{1}YdY=a\frac{1}XdX$$ which yields to $$\frac{dY}{dX}\frac{X}{Y}=a$$ Hence one simple interpretation of the coefficient $a$ will be the percent change in $Y$ for a percent change in $X$. This implies furthermore that the variable ...


13

There are multiple different types of correlation. The most common one is Pearson's correlation coefficient, which measures the amount of linear dependence between two vectors. That is, it essentially lays a straight line through the scatterplot and calculates its slope. This will of course change if you take logs! If you are interested in a measure of ...


13

One major advantage of log-differences is symmetry: if you have a log difference of $0.1$ today and one of $-0.1$ tomorrow, you are back from where you started. In contrast, 10% growth today and 10% decline tomorrow will not bring you back to the initial value.


13

I. Direct computation Gradshteyn & Ryzhik [1] (sect 4.358, 7th ed) list explicit closed forms for $$\int_0^\infty x^{\nu-1}e^{-\mu x}(\ln x)^p dx$$ for $p=2,3,4$ while the $p=1$ case is done in 4.352 (assuming you regard expressions in $\Gamma, \psi$ and $\zeta$ functions as closed form) -- from which it is definitely doable up to kurtosis; they give ...


13

Based on the size of your dataset, I suspect you are working with the single cell RNA-seq data. If so, I can confirm your observation: with scRNA-seq data, PCA explained variances after log-transform are typically much lower than beforehand. Here is a replication of your finding with the Tasic et al. 2016 dataset I have at hand: Here I used $\log(x+1)$ ...


12

Here's a version for dummies... We have the model $Y= \beta_o+\beta_1X+\varepsilon$ - a simple straight line through the data cloud - and we know that once we estimate the coefficients, a $1\text{-unit}$ increase in the prior value of $X=x_1$ will result in a increase of $\hat \beta_1$ in the value of $Y$, from $Y=y_1$, as $\hat\beta_1(x_1+1) -\hat\...


12

The map $$x \to \frac{x^p-1}{p}$$ is known in statistics as a Box-Cox transformation. The power $p$ may be any real number except $0$, for which the formula at the right is undefined (it would divide zero by zero). However, for any fixed positive $x$ the limit as $p$ approaches $0$ can readily be found by L'Hopital's Rule as $$\lim_{p\to 0} \frac{x^p-1}{p} ...


12

Except for very small counts, $\log(x)^2$ is essentially a linear function of $\log(x)$: The colored lines are least squares fits to $\log(x)^2$ vs $\log(x)$ for various ranges of counts $x$. They are extremely good once $x$ exceeds $10$ (and still awfully good even when $x\gt 4$ or so). Introducing the square of a variable sometimes is used to test ...


12

The moment generating function $M(t)$ of $Y=\ln X$ is helpful in this case, since it has a simple algebraic form. By the definition of m.g.f., we have $$\begin{aligned}M(t)&=\operatorname{E}[e^{t\ln X}]=\operatorname{E}[X^t]\\ &=\frac{1}{\Gamma(\alpha)\theta^\alpha}\int_0^\infty x^{\alpha+t-1}e^{-x/\theta}\,dx\\ &=\frac{\theta^{t}}{\Gamma(\alpha)}...


12

Almost. For a vector $U,$ the Yeo-Johnson with $\lambda=0$ is equivalent to the natural logarithm of $( U + 1 ).$


10

Transformation of an independent variable $X$ is one occasion where one can just be empirical without distorting inference as long as one is honest about the number of degrees of freedom in play. One way is to use regression splines for continuous $X$ not already known to act linearly. To me it's not a question of log vs. original scale; it's a question of ...


Only top voted, non community-wiki answers of a minimum length are eligible