# What is the purpose of characteristic functions?

I'm hoping that someone can explain, in layman's terms, what a characteristic function is and how it is used in practice. I've read that it is the Fourier transform of the pdf, so I guess I know what it is, but I still don't understand its purpose. If someone could provide an intuitive description of its purpose and perhaps an example of how it is typically used, that would be fantastic!

Just one last note: I have seen the Wikipedia page, but am apparently too dense to understand what is going on. What I'm looking for is an explanation that someone not immersed in the wonders of probability theory, say a computer scientist, could understand.

-

Back in the day, people used logarithm tables to multiply numbers faster. Why is this? Logarithms convert multiplication to addition, since $\log(ab) = \log(a) + \log(b)$. So in order to multiply two large numbers $a$ and $b$, you found their logarithms, added the logarithms, $z = \log(a) + \log(b)$, and then looked up $\exp(z)$ on another table.

Now, characteristic functions do a similar thing for probability distributions. Suppose $X$ has a distribution $f$ and $Y$ has a distribution $g$, and $X$ and $Y$ are independent. Then the distribution of $X+Y$ is the convolution of $f$ and $g$, $f * g$.

Now the characteristic function is an analogy of the "logarithm table trick" for convolution, since if $\phi_f$ is the characteristic function of $f$, then the following relation holds:

$$\phi_f \phi_g = \phi_{f * g}$$

Furthermore, also like in the case of logarithms,it is easy to find the inverse of the characteristic function: given $\phi_h$ where $h$ is an unknown density, we can obtain $h$ by the inverse Fourier transform of $\phi_h$.

The characteristic function converts convolution to multiplication for density functions the same way that logarithms convert multiplication into addition for numbers. Both transformations convert a relatively complicated operation into a relatively simple one.

-
Other items worth mentioned: (a) Recovery of moments via differentiation, (b) the fact that all distributions have characteristic functions (as compared to moment-generating functions), (c) The (essentially) one-to-one correspondence between distributions and their characteristic functions, and (d) the fact that many relatively common distributions have known characteristic functions but no known expression for the density (e.g., Levy stable distributions). –  cardinal Apr 17 '11 at 2:02
Good comments, @cardinal. Please consider turning them into an actual reply. –  whuber Apr 17 '11 at 16:55
For those of you who understand this topic, is it at all related to Characteristic Equations, as used with recurrence relations (i.e. in Knuth's Concrete Math)? My guess is that they're very different and only share the word "characteristic" by chance, but I thought I'd ask. –  Wayne Apr 18 '11 at 16:35

The purpose of characteristic functions is that they can be used to derive the properties of distributions in probability theory. If you're not interested in such derivations you do not need to learn about characteristic functions.

-
I suppose I might be interested in such derivations - I just don't quite get why we need to go to the characteristic function? Why is it easier than dealing directly with the pdf/cdf? –  Nick Apr 16 '11 at 21:10
@Nick This has a bit of folklore element, like "this is so elegant that this is THE representation of some distribution concept,...". Of course it helps with some maths, so it is not just a redundant toy, but for an everyday use it corresponds to a physicists forcing $\hbar$ to a classic problem just to use fine-structure constant. –  mbq Apr 16 '11 at 23:41
We don't need to use them. I said only that they can be used. Sometimes they give a quicker derivation, sometimes they're no help at all. Whether a derivation is 'easier' depends on what you already know - if you don't already know about characteristic functions it won't be easier. In some cases moment generating functions provide an alternative, and have a more direct interpretation. –  onestop Apr 17 '11 at 5:02