# From a statistical perspective: Fourier transform vs regression with Fourier basis

I am trying to understand whether discrete Fourier transform gives the same representation of a curve as a regression using Fourier basis. For example,

library(fda)
Y=daily$tempav[,1] ## my data length(Y) ## =365 ## create Fourier basis and estimate the coefficients mybasis=create.fourier.basis(c(0,365),365) basisMat=eval.basis(1:365,mybasis) regcoef=coef(lm(Y~basisMat-1)) ## using Fourier transform fftcoef=fft(Y) ## compare head(fftcoef) head(regcoef)  FFT gives a complex number, whereas regression gives a real number. Do they convey the same information? Is there a one to one map between the two sets of numbers? (I would appreciate if the answer is written from the statistician's perspective instead of the engineer's perspective. Many online materials I can find have engineering jargon all over the place, which makes them less palatable to me.) • I am not familiar with your code snippet, so cannot say if the following issue applies there. However, typically the DFT basis is defined in terms of integral ("whole number") frequencies, whereas a general "fourier basis" for regression could use arbitrary frequency ratios (e.g. including irrationals, at least in continuous arithmetic). This might also be of interest. Dec 2, 2016 at 4:45 • I think everybody would benefit if you write your question in math terms (as opposed to code snippets). What is the regression problem you solve? What are the Fourier basis functions you use? You'll be surprised at how answers to your question will improve. Jan 4, 2017 at 23:47 ## 3 Answers They're the same. Here's how... # Doing a Regression Say you fit the model $$y_t = \sum_{j=1}^n A_j \cos(2 \pi t [j/N] + \phi_j)$$ where$t=1,\ldots,N$and$n = \text{floor}(N/2)$. This isn't suitable for linear regression, though, so instead you use some trigonometry ($\cos(a + b) = \cos(a)\cos(b) - \sin(a)\sin(b)$) and fit the equivalent model: $$y_t = \sum_{j=1}^n \beta_{1,j} \cos(2 \pi t [j/N]) + \beta_{2,j}\sin(2 \pi t [j/N]).$$ Running linear regression on all of the Fourier frequencies$\{j/N : j = 1, \ldots, n \}$gives you a bunch ($2n$) of betas:$\{\hat{\beta}_{i,j}\}$,$i=1,2$. For any$j$, if you wanted to calculate the pair by hand, you could use: $$\hat{\beta}_{1,j} = \frac{\sum_{t=1}^N y_t \cos(2 \pi t [j/N])}{\sum_{t=1}^N \cos^2(2 \pi t [j/N])}$$ and $$\hat{\beta}_{2,j} = \frac{\sum_{t=1}^N y_t \sin(2 \pi t [j/N])}{\sum_{t=1}^N \sin^2(2 \pi t [j/N])}.$$ These are standard regression formulas. # Doing a Discrete Fourier Transform When you run a Fourier transform, you calculate, for$j=1, \ldots, n: \begin{align*} d(j/N) &= N^{-1/2}\sum_{t=1}^N y_t \exp[- 2 \pi i t [j/N]] \\ &= N^{-1/2}\left(\sum_{t=1}^N y_t\cos(2 \pi t [j/N]) - i \sum_{t=1}^N y_t\sin(2 \pi t [j/N])\right) . \end{align*} This is a complex number (notice thei$). To see why that equality holds, keep in mind that$e^{ix} = \cos(x) + i\sin(x)$,$\cos(-x) = \cos(x)$and$\sin(-x) = -\sin(x)$. For each$j$, taking the square of the complex conjugate gives you the "periodogram:" $$|d(j/N)|^2 = N^{-1}\left(\sum_{t=1}^N y_t\cos(2 \pi t [j/N])\right)^2 + N^{-1}\left(\sum_{t=1}^N y_t\sin(2 \pi t [j/N])\right)^2.$$ In R, calculating this vector would be I <- abs(fft(Y))^2/length(Y), which is sort of weird, because you have to scale it. Also you can define the "scaled periodogram" $$P(j/N) = \left(\frac{2}{N}\sum_{t=1}^N y_t \cos(2 \pi t [j/N]) \right)^2 + \left(\frac{2}{N}\sum_{t=1}^N y_t \sin(2 \pi t [j/N]) \right)^2.$$ Clearly$P(j/N) = \frac{4}{N}|d(j/N)|^2$. In R this would be P <- (4/length(Y))*I[(1:floor(length(Y)/2))]. # The Connection Between the Two It turns out the connection between the regression and the two periodograms is: $$P(j/N) = \hat{\beta}_{1,j}^2 + \hat{\beta}_{2,j}^2.$$ Why? Because the basis you chose is orthogonal/orthonormal. You can show for each$j$that$\sum_{t=1}^N \cos^2(2 \pi t [j/N]) = \sum_{t=1}^N \sin^2(2 \pi t [j/N]) = N/2$. Plug that in to the denominators of your formulas for the regression coefficients and voila. • +1 for the answer and the source. It would also be good if you can demonstrate the result with the R objects I posted. Jan 10, 2017 at 22:13 • @qoheleth I'll leave that to you. Just be weary of how fft() doesn't scale the way I wrote (I already mentioned this), that I haven't proved anything with intercepts, and that create.fourier.basis() scales the basis functions weirdly. Jan 12, 2017 at 22:40 They are strongly related. Your example is not reproducible because you didn't include your data, thus I'll make a new one. First of all, let's create a periodic function: T <- 10 omega <- 2*pi/T N <- 21 x <- seq(0, T, len = N) sum_sines_cosines <- function(x, omega){ sin(omega*x)+2*cos(2*omega*x)+3*sin(4*omega*x)+4*cos(4*omega*x) } Yper <- sum_sines_cosines(x, omega) Yper[N]-Yper[1] # numerically 0 x2 <- seq(0, T, len = 1000) Yper2 <- sum_sines_cosines(x2, omega) plot(x2, Yper2, col = "red", type = "l", xlab = "x", ylab = "Y") points(x, Yper)  Now, let's create a Fourier basis for regression. Note that, with$N = 2k+1$, it doesn't really make sense to create more than$N-2$basis functions, i.e.,$N-3=2(k-1)$non-constant sines and cosines, because higher frequency components are aliased on such a grid. For example, a sine of frequency$k\omega$is indistinguishable from a costant (sine): consider the case of$N=3$, i.e.,$k=1$. Anyway, if you want to double check, just change N-2 to N in the snippet below and look at the last two columns: you'll see that they're actually useless (and they create issues for the fit, because the design matrix is now singular). # Fourier Regression with fda library(fda) mybasis <- create.fourier.basis(c(0,T),N-2) basisMat <- eval.basis(x, mybasis) FDA_regression <- lm(Yper ~ basisMat-1) FDA_coef <-coef(FDA_regression) barplot(FDA_coef)  Note that the frequencies are exactly the right ones, but the amplitudes of nonzero components are not (1,2,3,4). The reason is that the fda Fourier basis functions are scaled in a weird way: their maximum value is not 1, as it would be for the usual Fourier basis${1,\sin{ \omega x},\cos{ \omega x},\dots}$. It's not$\frac{1}{\sqrt \pi}$either, as it would have been for the orthonormal Fourier basis,${\frac{1}{\sqrt {2\pi}},\frac{\sin{ \omega x}}{\sqrt \pi},\frac{\cos{ \omega x}}{\sqrt \pi},\dots}$. # FDA basis has a weird scaling max(abs(basisMat)) plot(mybasis)  You clearly see that: 1. the maximum value is less than$\frac{1}{\sqrt \pi}$2. the Fourier basis (truncated to the first$N-2$terms) contains a constant function (the black line), sines of increasing frequency (the curves which are equal to 0 at the domain boundaries) and cosines of increasing frequency (the curves which are equal to 1 at the domain boundaries), as it should be Simply scaling the Fourier basis given by fda, so that the usual Fourier basis is obtained, leads to regression coefficients having the expected values: basisMat <- basisMat/max(abs(basisMat)) FDA_regression <- lm(Yper ~ basisMat-1) FDA_coef <-coef(FDA_regression) barplot(FDA_coef, names.arg = colnames(basisMat), main = "rescaled FDA coefficients")  Let's try fft now: note that since Yper is a periodic sequence, the last point doesn't really add any information (the DFT of a sequence is always periodic). Thus we can discard the last point when computing the FFT. Also, the FFT is just a fast numerical algorithm to compute the DFT, and the DFT of a sequence of real or complex numbers is complex. Thus, we really want the moduluses of the FFT coefficients: # FFT fft_coef <- Mod(fft(Yper[1:(N-1)]))*2/(N-1)  We multiply by$\frac{2}{N-1}$in order to have the same scaling as with the Fourier basis${1,\sin{ \omega x},\cos{ \omega x},\dots}$. If we didn't scale, we would still recover the correct frequencies, but the amplitudes would all be scaled by the same factor with respect to what we found before. Let's now plot the fft coefficients: fft_coef <- fft_coef[1:((N-1)/2)] terms <- paste0("exp",seq(0,(N-1)/2-1)) barplot(fft_coef, names.arg = terms, main = "FFT coefficients")  Ok: the frequencies are correct, but note that now the basis functions are not sines and cosines any more (they're complex exponentials$\exp{ni\omega x}$, where with$i$I denote the imaginary unit). Note also that instead than a set of nonzero frequencies (1,2,3,4) as before, we got a set (1,2,5). The reason is that a term$x_n\exp{ni\omega x}$in this complex coefficient expansion (thus$x_n$is complex) corresponds to two real terms$a_n sin(n\omega x)+b_n cos(n\omega x)$in the trigonometric basis expansion, because of the Euler formula$\exp{ix}=\cos{x}+i\sin{x}$. The modulus of the complex coefficient is equal to the sum in quadrature of the two real coefficients, i.e.,$|x_n|=\sqrt{a_n^2+b_n^2}$. As a matter of fact,$5=\sqrt{3^3+4^2}\$.

• thanks DeltaIV, the data daily comes with the fda package. Jan 8, 2017 at 22:22
• @qoheleth I didn't know. This evening I will modify my answer using your dataset, and I will clarify a couple points. Jan 9, 2017 at 7:36

I want to say that the first answer has an overflow error since the second formula for DFT coefficients is infinite when j = N/2 at this value the argumment for sine is π * t making the value zero for all t's. β^2,j=∑Nt=1ytsin(2πt[j/N])∑Nt=1sin2(2πt[j/N]) There are others DFT Data Regressions in published papers out there in google but i've not analyzed them correctly as they are confusing me. Perhaps the author of the first answer may help us as I'm tanglemessed up with this. Thanks in advance