2 added 272 characters in body
source | link

I'm performing an harmonic fit to data I know (from physical constraints) come from a periodic source of the form $$\sum_j^M \sum_i^N a_{i,j}\sin(2\pi f_it)+b_{i,j}\cos(2\pi f_it)$$ using the Lomb-Scargle periodogram (I know this is not the optimal thing to do, but it's fast and reliable so far), where I'm also trying to estimate $N$ and $M$, the number of frequencies and the number of harmonics per frequency to fit. The number of harmonics is really not a very hard problem to solve once the number of frequencies is fixed, because I can perform methods like Ridge Regression, LASSO, etc. in order to find the best subset of harmonics, so my problem is really on estimating the optimal number of frequencies.

The thing is that I know red noise is almost always present, and certain harmonic fits give AR-like spectra on the residuals (and, at least to me, it seems like I could even have ARMA-like spectra). The "classical" Lomb-Scargle periodogram, on the other hand, assumes white noise in order to test the significance of the peaks, so the significance test on the peaks doesn't seem like a good idea in order to test if there are any residual frequencies on the periodogram.

I've been trying to derive significance tests assuming ARMA-like noise, but the problem of defining the order $p,q$ of the process arises: do you know of a way of differencing between correlated noise and sine wave-like spectra? This paper shows a way of doing this assuming AR(1) noise (in fact something more like a CAR-type noise?), but the problem of it is that it actually knows there is red noise. I think I can "see" how to do it: sine wave-like spectra has almost-symmetric decreasing peaks around the main peak because of the (uneven) sampling, which ARMA-like noise doesn't have (the amplitudes are random with an apparent upper limit dictated by the form of the spectra). I've also been thinking in performing fits with ARMA and sine-like waves, and perform model selection via AIC...any suggestions?

I'm performing an harmonic fit to data I know (from physical constraints) come from a periodic source of the form $$\sum_j^M \sum_i^N a_{i,j}\sin(2\pi f_it)+b_{i,j}\cos(2\pi f_it)$$ using the Lomb-Scargle periodogram (I know this is not the optimal thing to do, but it's fast and reliable so far), where I'm also trying to estimate $N$ and $M$, the number of frequencies and the number of harmonics per frequency to fit. The number of harmonics is really not a very hard problem to solve once the number of frequencies is fixed, because I can perform methods like Ridge Regression, LASSO, etc. in order to find the best subset of harmonics, so my problem is really on estimating the optimal number of frequencies.

The thing is that I know red noise is almost always present, and certain harmonic fits give AR-like spectra on the residuals (and, at least to me, it seems like I could even have ARMA-like spectra). The "classical" Lomb-Scargle periodogram, on the other hand, assumes white noise in order to test the significance of the peaks, so the significance test on the peaks doesn't seem like a good idea in order to test if there are any residual frequencies on the periodogram.

I've been trying to derive significance tests assuming ARMA-like noise, but the problem of defining the order $p,q$ of the process arises: do you know of a way of differencing between correlated noise and sine wave-like spectra? I think I can "see" it: sine wave-like spectra has almost-symmetric decreasing peaks around the main peak because of the (uneven) sampling, which ARMA-like noise doesn't have (the amplitudes are random with an apparent upper limit dictated by the form of the spectra). I've also been thinking in performing fits with ARMA and sine-like waves, and perform model selection via AIC...any suggestions?

I'm performing an harmonic fit to data I know (from physical constraints) come from a periodic source of the form $$\sum_j^M \sum_i^N a_{i,j}\sin(2\pi f_it)+b_{i,j}\cos(2\pi f_it)$$ using the Lomb-Scargle periodogram (I know this is not the optimal thing to do, but it's fast and reliable so far), where I'm also trying to estimate $N$ and $M$, the number of frequencies and the number of harmonics per frequency to fit. The number of harmonics is really not a very hard problem to solve once the number of frequencies is fixed, because I can perform methods like Ridge Regression, LASSO, etc. in order to find the best subset of harmonics, so my problem is really on estimating the optimal number of frequencies.

The thing is that I know red noise is almost always present, and certain harmonic fits give AR-like spectra on the residuals (and, at least to me, it seems like I could even have ARMA-like spectra). The "classical" Lomb-Scargle periodogram, on the other hand, assumes white noise in order to test the significance of the peaks, so the significance test on the peaks doesn't seem like a good idea in order to test if there are any residual frequencies on the periodogram.

I've been trying to derive significance tests assuming ARMA-like noise, but the problem of defining the order $p,q$ of the process arises: do you know of a way of differencing between correlated noise and sine wave-like spectra? This paper shows a way of doing this assuming AR(1) noise (in fact something more like a CAR-type noise?), but the problem of it is that it actually knows there is red noise. I think I can "see" how to do it: sine wave-like spectra has almost-symmetric decreasing peaks around the main peak because of the (uneven) sampling, which ARMA-like noise doesn't have (the amplitudes are random with an apparent upper limit dictated by the form of the spectra). I've also been thinking in performing fits with ARMA and sine-like waves, and perform model selection via AIC...any suggestions?

1
source | link

Testing periodogram "peaks": sine-like wave or AR/MA/ARMA noise?

I'm performing an harmonic fit to data I know (from physical constraints) come from a periodic source of the form $$\sum_j^M \sum_i^N a_{i,j}\sin(2\pi f_it)+b_{i,j}\cos(2\pi f_it)$$ using the Lomb-Scargle periodogram (I know this is not the optimal thing to do, but it's fast and reliable so far), where I'm also trying to estimate $N$ and $M$, the number of frequencies and the number of harmonics per frequency to fit. The number of harmonics is really not a very hard problem to solve once the number of frequencies is fixed, because I can perform methods like Ridge Regression, LASSO, etc. in order to find the best subset of harmonics, so my problem is really on estimating the optimal number of frequencies.

The thing is that I know red noise is almost always present, and certain harmonic fits give AR-like spectra on the residuals (and, at least to me, it seems like I could even have ARMA-like spectra). The "classical" Lomb-Scargle periodogram, on the other hand, assumes white noise in order to test the significance of the peaks, so the significance test on the peaks doesn't seem like a good idea in order to test if there are any residual frequencies on the periodogram.

I've been trying to derive significance tests assuming ARMA-like noise, but the problem of defining the order $p,q$ of the process arises: do you know of a way of differencing between correlated noise and sine wave-like spectra? I think I can "see" it: sine wave-like spectra has almost-symmetric decreasing peaks around the main peak because of the (uneven) sampling, which ARMA-like noise doesn't have (the amplitudes are random with an apparent upper limit dictated by the form of the spectra). I've also been thinking in performing fits with ARMA and sine-like waves, and perform model selection via AIC...any suggestions?