Let $\phi_{x}(t)= E [ e^{itx}]$ be the characteristic function

If X is a continuous random variable, then:

$\phi_{x}(t)= E [ e^{itx}] = \int e^{itx} f(x)dx$ (being $f(x)$ the probability density function of x)

If X is a discrete random variable, then:

$\phi_{x}(t)= E [ e^{itx}] = \sum_{k} e^{itx_{k}} p(x_{k})$ (being $p(x)$ the probability mass function of x)

To be general Fourier transform can be define as (Ref: https://www.johndcook.com/blog/fourier-theorems/):

$F(s)=\frac{1}{A}\int_{-\infty}^{\infty} e^{iBst}f(t)dt$

The choices that are found in practice are:

$A=\sqrt{2\pi}, B=\pm 1;$ $A=1, B=\pm 2\pi;$ $A=1, B=\pm 1$

Choosing A=1 and B=1 to relate it to the characteristic function, the inverse Fourier transform is:

$f(t)=\frac{1}{2\pi}\int_{-\infty}^{\infty} e^{-ist}F(s)ds$

In continuous case:

$\phi_{x}(t) = \int e^{itx} f(x)dx \rightarrow f(x) = \frac{1}{2\pi}\int_{-\infty}^{\infty} e^{-ixt}\phi_{x}(t)dt$

That is the utility of the characteristic function, it allows me to know the probability function

But in the discrete case I get problems:

If discrete Fourier transform (DFT) is :

$F(s)=\sum_{n=0}^{N-1} e^{-2\pi sn/N}f(n)$

The inverse DFT is:

$f(t)=\frac{1}{N}\sum_{n=0}^{N-1} e^{2\pi sn/N}F(s)$

So, Would the "inverse" characteristic function be

$\phi_{x}(t) = \sum_{k} e^{itx_{k}} p(x_{k}) \rightarrow P(x) = \frac{1}{2\pi N}\sum_{k} e^{-ixt_{k}}\phi_{x}(t_{n})$ ?

So if FT can have different definitions for the continuous case. What happen with the discrete case? How many definitions are?


1 Answer 1


For the discrete case, you need to look at DTFT, not DFT. N-point DFT assumes that the underlying function is periodic, which is not the case for probability mass functions. You can then apply similar logic for inverse transform:

$$\phi_{x}(t)= E [ e^{itx}] = \sum_{k} e^{itx_{k}} p(x_{k})\rightarrow p(x)=\frac{1}{2\pi}\int_0^{2\pi}e^{-ixt}\phi_x(t)dt$$

For example, characteristic function for Bernouilli RV is $\phi_x(t)=1-p+pe^{it}$. Applying the formula yields the following:

$$\begin{align}p(x)&={1\over 2\pi}\int_0^{2\pi}e^{-ixt} (1-p+pe^{it})dt\\&=(1-p){1\over 2\pi}\underbrace{\int_0^{2\pi} e^{-itx}dt}_{2\pi\delta_x} + p\frac{1}{2\pi}\underbrace{\int_0^{2\pi}e^{-it(x-1)}dt}_{2\pi\delta_{x-1}}\\&=(1-p)\delta_x+p\delta_{x-1}\end{align}$$

where $\delta_x$ is the Kronecker delta function, i.e. it's $1$ if $x=0$, and $0$ otherwise.

This is from Fourier transform perspective. Probability theory has its own principled way of calculating inverses.

  • 1
    $\begingroup$ +1 and can you provide more details may be with an example? $\endgroup$
    – Haitao Du
    Jul 7, 2020 at 9:42
  • $\begingroup$ @HaitaoDu great advice. I've added an example for Bernoulli RV. $\endgroup$
    – gunes
    Jul 7, 2020 at 10:46
  • $\begingroup$ @gunes Why is the upper limit of the integral 2$\pi$? Also I know I can use Stieltjes integral as $F(b)-F(a)=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty \frac{ e^{-itb}- e^{-ita}}{-it}f(t)dt$ But if $F_{x}(x)=P(X\leq x) = \sum_{x_{i} \leq x} P(X=x_{i}) = \sum_{x_{i} \leq x} p(x_{i})$ How do you integrate it and put inside the integral? $\endgroup$
    – Ilya_Curie
    Jul 7, 2020 at 16:52
  • $\begingroup$ @gunus And also I think $\delta_{x}$ is Dirac delta no Kronecker delta $\endgroup$
    – Ilya_Curie
    Jul 7, 2020 at 18:35
  • 1
    $\begingroup$ for a probability theory treatment, this source seems good: statweb.stanford.edu/~adembo/stat-310b/lnotes.pdf for fourier transform review, I strongly recommend Oppenheim $\endgroup$
    – gunes
    Jul 7, 2020 at 19:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.