Calculating right values of Periodogram using Fourier Analysis

Question

In the book, Economic Cycles: There Law and Cause By Henry Ludwell Moore, he plots Periodogram of rainfall of Ohio valley. He uses 72 years data (1839-1910) and tries to find the most dominant cycle in that time frame.

I was trying to calculate those Periodogram values, but have failed to do so. After applying the Fourier Analysis formula I was getting right coefficient values for cycles which completely divided 72, such as 3, 4, 6, 8, 9, 12, 18, 24 and 36. For all other cycles, I was getting wrong values.

Formula I am using to calculate coefficients (in excel):

a = (2/N)*(R)*Cos(2πt/T)

b = (2/N)*(R)*Sin(2πt/T)

T = no. of cycle from 3 to 36, N = total observations in T cycle, R = rainfall data, t = number ranging from 0 to 71.

I asked this question on another site, where someone was kind enough to suggest me this site. He also stated that the problem I was facing was due to boundary effect. Therefore, I should discard, as an example, data 71 and 72 for period 7. His solution worked and I was able to get coefficient values close to Moore's values.

However, when data points increase from 72 to lets say 500, and T increases from 36 to 150 or 200. I am facing the same problem. The Periodogram values are repeating like waves and I am unable to spot the dominant cycle.

I have been reading a lot about Fourier Analysis and Periodogram for the past few days, but because I don't have good understanding of higher mathematics it has not yielded any result. I am still struggling to find the right answer. Hoping someone could help me here.

Thanks

Link of Excel file

Answer 1

First of all let us clear up this mess from the book:

What the book did: Take some input $(f(0), f(1), ..., f(M-1))$ i.e. we start with $M$ values. Take different 'possible periods' $T=3,4,5,...$ and for every possible period choose the maximal number $N_T$ such that $N_T \leq M$ and $T$ divides $N_T$. The consider the elements $(f(0), ..., f(N_T - 1))$ so that you get $N_T$ elements of the vector.

For every possible period $T$, put $\kappa_T = \frac{2\pi}{T}$ and compute the discrete real Fourier transform, i.e. get numbers $(a_0^{(T)}, a_1^{(T)}, b_1^{(T)}, a_2^{(T)}, b_2^{(T)}, ..., a_{T-1}^{(T)}, b_{T-1}^{(T)})$ such that

\begin{align*} f(t) \approx a_0^{(T)} & + a_1^{(T)} \cos(1*\kappa_T t) + a_2^{(T)} \cos(2*\kappa_T t) + ... + a_{T-1}^{(T)} \cos([T-1]*\kappa_T t) \\ & + b_1^{(T)} \sin(1*\kappa_T t) + b_2^{(T)} \sin(2*\kappa_T t) + ... + b_{T-1}^{(T)} \sin([T-1]*\kappa_T t) \end{align*}

Now the periodogram in the book you refer to at 'possible frequency' $T$ is defined as $$P(T) = (a_1^{(T)})^2 + (b_1^{(T)})^2$$

i.e. the size of the first coefficient in a Fourier transform that varies along the frequency.

First of all this seems weird, because usually, people define the periodogram in terms of one fixed Fourier transform as a function on the frequencies. However, notice that in the book, the author does exactly the same thing, just expressed in a very quirky way, i.e. (if $T|M$) $$a_1^{(T)} \approx a_T{(M)} ~\text{and}~ b_1^{(T)} \approx b_T{(M)}$$

i.e. The first Fourier coefficient of the Fourier transform with a 'possible period' $T$ is nothing else but the $T$-th Fourier coefficient in one single, fixed Fourier transform with the largest posible frequency $M$

so that we are really dealing with the 'usual' Periodogram.

Now let us consider the single Fourier transform with maximal period $M$:

\begin{align*} f(t) \approx a_0 & + a_1 \cos(1*\kappa t) + a_2 \cos(2*\kappa t) + ... + a_{M-1} \cos([M-1]*\kappa t) \\ & + b_1 \sin(1*\kappa t) + b_2 \sin(2*\kappa t) + ... + b_{M-1} \sin([M-1]*\kappa t) \end{align*}

What does the size $P(w) = a_w^2 + b_w^2$ of the $w$-the Fourier coefficients tell us about the input vector? It tells us 'how much the input vector is $w$-periodical'. However, then it is not surprising that the sizes repeat in cycles: Consider an input that is $3$ periodic. What else can we say about this input? Is it $5$-periodic as well? We don't know but one thing we can say for sure: It is clearly $6$-periodic because $f(t+6) = f([t+3]+3) = f(t+3) = f(t)$ as $f$ is $3$-periodic. Analogously, $f$ is definitely $k*3$-periodic for every $k$! Hence, the 'periodicity' decomposes into 'prime' frequencies (the first, smallest peak that you observe in the Periodogram) and 'composed' frequencies (all multiples of these 'prime' frequencies).

Hence, the 'true' frequency that you search for should be the minimal peak in the Periodogram.

For more information you need to share the shape of you rdata and/or your data and at least the Periodogram that you computed.