I have seen in multiple sources that if we start with writing a time series as follows (assuming $n$ is odd):

$$x_t = a_o + \sum_{j=1}^{(n-1)/2}{[a_jcos(2\pi tj/n) + b_jsin(2\pi tj/n)]}$$

Then we will be able to estimate the $a_j$'s and $b_j$'s by treating it as a linear regression problem. Specifically, (letting $z_{tj}=cos(2\pi tj/n)$ or $z_{tj}=sin(2\pi tj/n)$, we can write

$$a_j = \frac{\sum_{t=1}^nx_tz_{t,j}}{\sum_{t=1}^nz^2_{t,j}}$$

and $$b_j = \frac{\sum_{t=1}^nx_tz_{t,j}}{\sum_{t=1}^nz^2_{t,j}}$$

This is where I get stuck. I understand that the above formulations come from estimating the coefficients of the linear regression, however, the denominator of each coefficient confuses me.

Given that this is a multiple linear regression problem, I've been trying to work through it using vectors/matrices. That is, using


where $\hat{\beta}$ is the vector of coefficients (in our case this will represent the $a_j$'s and $b_j$'s) and $z_t$ is the vector of $cos(2\pi tj/n)$'s and $sin(2\pi tj/n)$'s, which we are regressing on.

By letting $q=n-1$ arbitrarily and $z_t = [1, z_{t1}, ... z_{tq}]'$, we can work out the numerator of $a_j$ and $b_j$ as follows

$$\sum_{t=1}^nz_tx_t=\sum_{t=1}^n[1, z_{t1}, ... z_{t1}]'x_t$$ $$=[\sum_{t=1}^nx_t, \sum_{t=1}^nz_{t1}x_t, ...\sum_{t=1}^nz_{tq}x_t]$$

The first element is $a_o$ and every other element is the numerator of the corresponding coefficient it is estimating.

Then, to calculate the denominator, we write

$$\sum^n_{t=1}z_tz_t'= \begin{bmatrix} 1 & z_{t1} & ... & z_{tq} \\ z_{t1} & z_{t1}^2 & ... & z_{t1}z_{tq} \\ ... & ... & & ...\\ z_{tq} & z_{t1}z_{tq} & ... & z_{tq}^2 \\ \end{bmatrix} $$

From here we take the inverse of this matrix, but I fail to see how this would equate to $\sum_{t=1}^nz^2_{t,j}$ being the denominator of each coefficient. Unless there is some serious math involved, I don't see how we would be able to generalise the results of this inverse matrix so succinctly.

If there is more advanced math that I am missing, I imagine the matrix being symmetric has something to do with, as this would mean the inverse is also symmetric, but that's as far as I can get. The only thing I can think of is that I am overlooking an important piece of information before I got to this point, which is probably more likely.

  • 2
    $\begingroup$ The part you are missing is the Orthogonality of the $\sin$ and $\cos$ basis functions. Because of it the off-diagonal elements of your matrix vanish, so inverting it gives the inverses of the diagonal elements $\endgroup$
    – J. Delaney
    May 3, 2022 at 13:13
  • $\begingroup$ Thanks for your comment. The only thing I'm struggling with is establishing the orthogonality of sin and cos. From the link provided, it seems e^ix = cos(x) + isin(x) is an orthogonal basis, but not necessarily the individual sin and cos. This is a problem (for me anyway) as the ztj represent individual sin and cos functions. $\endgroup$
    – benja616
    May 4, 2022 at 10:21
  • $\begingroup$ That's tight, but you can get to the individual sin and cos functions by separately considering the real and imaginary parts and using a little trigonometry, See e.g. here or here $\endgroup$
    – J. Delaney
    May 4, 2022 at 12:04
  • $\begingroup$ Thanks for the resources. I think I've worked it out. I believe I made a mistake in working out the matrix for the zt*zt', in that I forgot to include the summation in the final equation statement. Correct me if I'm wrong, but we are summing n matrices (which I forgot to write), which allows the sin and cos functions to become orthogonal. The diagonal elements don't become orthogonal because they share the same frequency. Also, and I'm unsure here, we are assuming that the summation covers an integer multiple of the oscillation, otherwise the sums of cos couldn't equal 0. $\endgroup$
    – benja616
    May 5, 2022 at 8:50
  • $\begingroup$ Also, if you leave your comments as an answer, I am happy to accept it. $\endgroup$
    – benja616
    May 5, 2022 at 8:52


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.