Characteristic function and Fourier transform for a discrete random variable!

Question

Let $\phi_{x}(t)= E [ e^{itx}]$ be the characteristic function

If X is a continuous random variable, then:

$\phi_{x}(t)= E [ e^{itx}] = \int e^{itx} f(x)dx$ (being $f(x)$ the probability density function of x)

If X is a discrete random variable, then:

$\phi_{x}(t)= E [ e^{itx}] = \sum_{k} e^{itx_{k}} p(x_{k})$ (being $p(x)$ the probability mass function of x)

To be general Fourier transform can be define as (Ref: https://www.johndcook.com/blog/fourier-theorems/):

$F(s)=\frac{1}{A}\int_{-\infty}^{\infty} e^{iBst}f(t)dt$

The choices that are found in practice are:

$A=\sqrt{2\pi}, B=\pm 1;$ $A=1, B=\pm 2\pi;$ $A=1, B=\pm 1$

Choosing A=1 and B=1 to relate it to the characteristic function, the inverse Fourier transform is:

$f(t)=\frac{1}{2\pi}\int_{-\infty}^{\infty} e^{-ist}F(s)ds$

In continuous case:

$\phi_{x}(t) = \int e^{itx} f(x)dx \rightarrow f(x) = \frac{1}{2\pi}\int_{-\infty}^{\infty} e^{-ixt}\phi_{x}(t)dt$

That is the utility of the characteristic function, it allows me to know the probability function

But in the discrete case I get problems:

If discrete Fourier transform (DFT) is :

$F(s)=\sum_{n=0}^{N-1} e^{-2\pi sn/N}f(n)$

The inverse DFT is:

$f(t)=\frac{1}{N}\sum_{n=0}^{N-1} e^{2\pi sn/N}F(s)$

So, Would the "inverse" characteristic function be

$\phi_{x}(t) = \sum_{k} e^{itx_{k}} p(x_{k}) \rightarrow P(x) = \frac{1}{2\pi N}\sum_{k} e^{-ixt_{k}}\phi_{x}(t_{n})$ ?

So if FT can have different definitions for the continuous case. What happen with the discrete case? How many definitions are?

Answer 1

For the discrete case, you need to look at DTFT, not DFT. N-point DFT assumes that the underlying function is periodic, which is not the case for probability mass functions. You can then apply similar logic for inverse transform:

$$\phi_{x}(t)= E [ e^{itx}] = \sum_{k} e^{itx_{k}} p(x_{k})\rightarrow p(x)=\frac{1}{2\pi}\int_0^{2\pi}e^{-ixt}\phi_x(t)dt$$

For example, characteristic function for Bernouilli RV is $\phi_x(t)=1-p+pe^{it}$. Applying the formula yields the following:

$$\begin{align}p(x)&={1\over 2\pi}\int_0^{2\pi}e^{-ixt} (1-p+pe^{it})dt\\&=(1-p){1\over 2\pi}\underbrace{\int_0^{2\pi} e^{-itx}dt}_{2\pi\delta_x} + p\frac{1}{2\pi}\underbrace{\int_0^{2\pi}e^{-it(x-1)}dt}_{2\pi\delta_{x-1}}\\&=(1-p)\delta_x+p\delta_{x-1}\end{align}$$

where $\delta_x$ is the Kronecker delta function, i.e. it's $1$ if $x=0$, and $0$ otherwise.

This is from Fourier transform perspective. Probability theory has its own principled way of calculating inverses.