Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

This question came in the context of understanding how to get a distribution of a sum of two iid random variables. I'm working through the top answer to this question Consider the sum of $n$ uniform distributions on $[0,1]$, or $Z_n$. Why does the cusp in the PDF of $Z_n$ disappear for $n \geq 3$?, trying to understand why characteristic functions behave the way they do.

The title is the whole question.

share|improve this question
up vote 11 down vote accepted

Let's suppose we could find a (measurable) function $\chi$ defined on the real numbers with the property that

$$\chi(a + b) = \chi(a)\chi(b)$$

for all numbers $a$ and $b$ and for which there is a finite positive number $M$ for which $|\chi(a)| \le M$ for all $a$. Notice how $\chi$ relates addition (which is the fundamental operation appearing in a convolution) and multiplication.

Why are these properties useful? Suppose $X$ and $Y$ are independent random variables. Let $t$ be any real number. Then (taking up these two properties in reverse order)

  1. $\mathbb{E}(\chi(tX)) \le \mathbb{E}(|\chi(tX)|) = \mathbb{E}(M) = M \lt \infty$ (with a similar expression for $Y$) shows that the expectations of the random variables $\chi(tX)$ and $\chi(tY)$ exist and are finite, with a uniform bound $M$ independent of $t$.

    This procedure of taking a random variable $X$ and transforming it into the function

    $$t \to \mathbb{E}(\chi(tX)) = (\operatorname{cf}_\chi(X))(t)$$

    thereby assigns a well-defined, bounded function $\operatorname{cf}_\chi(X)$ to every random variable $X$--no matter what awful properties $X$ might have.

  2. $\mathbb{E}(\chi(t(X+Y))) = \mathbb{E}(\chi(tX)\chi(tY)) = \mathbb{E}(\chi(tX))\mathbb{E}(\chi(tY))$ because $X$ and $Y$ are independent. Written slightly differently,

    $$(\operatorname{cf}_\chi(X+Y))(t) = ((\operatorname{cf}_\chi(X))(\operatorname{cf}_\chi(Y)))(t)$$

    That is, the transformation $\operatorname{cf}_\chi$ converts convolution (addition of random variables) into (pointwise) multiplication of functions.

Much more can be said: see the literature on Fourier Analysis. But in the meantime, the question has been answered in a way that shows "time" and "frequency" may be red herrings: this fundamental property of converting convolution into multiplication relies only on the existence of a nice $\chi$.

The only real-valued functions with the defining properties of $\chi$ are $\chi(a) = 1$ and $\chi(a) = 0$. They lead to nothing useful. But if we allow $\chi$ to have complex values, then $\chi(a) = \exp(ia)$ is one such function and it produces useful results. (Moreover, all such $\chi$ are derived from this one: they must be of the form $a\to \exp(ia\lambda)$ for some fixed real number $\lambda$.) In this case $\operatorname{cf}_\chi(X)$ is called the characteristic function of $X$.


It's not hard to see that when $\chi$ is not identically zero, $|\chi(a)|$ must always equal $1$, no matter what $a$ is. Such functions are called (complex) multiplicative characters (of the additive group of real numbers).

share|improve this answer
    
+1 I wish everyone introduced this concept the same way as you... – Mehrdad Feb 9 at 3:58

The answer depends on what you're looking for in the answer.

For instance, to me this was the answer, and the rest was just the details: $$e^a\cdot e^b=e^{a+b}$$ It's all about the exponent, when you multiply them their arguments get added.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.