3
$\begingroup$

I've got an equation that contains

$$x^p - 1$$

$x$ is any positive number (such as 2) and $p$ is a small positive number close to 0 (such as 0.001).

For some reason (that I may have known in High School), this can be approximated by

$$\ln(x) \times p$$

Can anyone explain why this approximation works? Does it have something to do with approximating a natural log with a series?

$\endgroup$
12
$\begingroup$

The map $$x \to \frac{x^p-1}{p}$$ is known in statistics as a Box-Cox transformation. The power $p$ may be any real number except $0$, for which the formula at the right is undefined (it would divide zero by zero). However, for any fixed positive $x$ the limit as $p$ approaches $0$ can readily be found by L'Hopital's Rule as

$$\lim_{p\to 0} \frac{x^p-1}{p} = \lim_{p\to 0} \frac{\frac{d}{dp}(x^p-1)}{\frac{d}{dp}(p)} = \lim_{p\to 0} \frac{\log(x) \exp(p\log x)}{1} = \log(x) $$

where I computed

$$\frac{d}{dp} x^p = \frac{d}{dp} \exp(p\log x) = \log(x) \exp(p\log x)$$ using the Chain Rule and exploited the continuity of $\exp$ to find the limit simply by setting $p=0$ in the right hand side.

This establishes $p=0$ as a removable singularity in the Box-Cox family and helps us understand the "zero power" as naturally corresponding to the logarithm.


Edit

Alternatively, recognize that this transformation can also be expressed as

$$x \to \int_1^x y^{p-1}\mathrm{d}y.\tag{*}$$

Simply set $p=0$ to obtain

$$x \to \int_1^x y^{-1}\mathrm{d}y = \log(x)$$

(because that's the definition of the natural logarithm).

So, to answer the question, notice that the function $p \to y^{p-1} = \exp((p-1)\log(y))$ is continuous because $\exp$ is continuous. Therefore the integral $(*)$ is a continuous (even a differentiable) function of $p,$ whence when $p\approx 0,$ the transformation will be approximated by $p=0:$ the logarithm. The closeness of the approximation, though, varies with $y,$ getting worse as $y\to 0$ or $y\to \infty.$

$\endgroup$
3
$\begingroup$

I have nothing to add to whuber's brilliant answer using the Box-Cox transformation. I just wanted to offer an alternative source of the approximation, using the Maclaurin series for the natural logarithm:

$$\ln (y+1) = y - \frac{y^2}{2} + \frac{y^3}{3} - \cdots.$$

Ignoring the higher-order terms in the expansion gives the crude first-order approximation:

$$\ln (y+1) \approx y.$$

Substituting $y=x^p-1$ gives the approximation in your question:

$$p \ln(x) = \ln(x^p) = \ln(y+1) \approx y = x^p-1.$$

Thus, in addition to being derivable from the Box-Cox transform, this approximation can be seen as a crude first-order Maclaurin approximation to the logarithm (which is indeed an approximation you might have run across in High School).

$\endgroup$
  • 1
    $\begingroup$ (+1), Although your approach is equivalent to L'Hopital's Rule, and therefore isn't really different, it suggests one might invert the log and analyze the exponential. Indeed, because $x^p$ is defined to be $\exp(p\log x),$ use the Maclaurin series for $\exp$ (which is a universal way of defining the exponential) to conclude that $$x^p-1 = \exp(p\log x) - 1 = p\log x + O(|p\log x|^2).$$ The beauty of this approach lies in its generality: it will, for example, work in any normed commutative algebra, such as that of square matrices $x.$ $\endgroup$ – whuber Jan 8 at 20:42
  • $\begingroup$ I hesitate to quibble with @whuber on anything, but here goes. $x^p$ is well defined for some negative $x$. Take $p = 1/3$: the cube root of a negative number is well defined. You then have a difficulty with treating that as $\exp(p \log x)$. As I understand it, powering code is often based on first taking logarithms, so that many programs won't give you $(-8)^{1/3}$ without you reaching in, ignoring the minus sign and putting it back at the end. The problem is that logarithm step is just trapped for negative arguments. (The complex number interpretation is I think no refutation.) $\endgroup$ – Nick Cox Jan 8 at 22:29
  • $\begingroup$ @Nick The complex number interpretation is the refutation. The "well defined" assertion requires a limited definition of powers (it applies only to special exponents), one that ultimately is abandoned in favor of starting either with $\log z = \int_1^z \mathrm{d}z/z$ or $\exp(z) = 1 + z + z^2/2! +\ldots,$ defining the other as its inverse, and going on from there in order to make sense of real powers of all real numbers, positive and negative. See, for instance, the introductory chapter to Rudin's Real and Complex Analysis. (What computers actually do is irrelevant.) $\endgroup$ – whuber Jan 8 at 23:28
  • $\begingroup$ Thanks for the detail. I should have followed prudence, not the math[s] I remember. $\endgroup$ – Nick Cox Jan 8 at 23:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.