Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'm performing an harmonic fit to data I know (from physical constraints) come from a periodic source of the form $$\sum_j^M \sum_i^N a_{i,j}\sin(2\pi f_it)+b_{i,j}\cos(2\pi f_it)$$ using the Lomb-Scargle periodogram (I know this is not the optimal thing to do, but it's fast and reliable so far), where I'm also trying to estimate $N$ and $M$, the number of frequencies and the number of harmonics per frequency to fit. The number of harmonics is really not a very hard problem to solve once the number of frequencies is fixed, because I can perform methods like Ridge Regression, LASSO, etc. in order to find the best subset of harmonics, so my problem is really on estimating the optimal number of frequencies.

The thing is that I know red noise is almost always present, and certain harmonic fits give AR-like spectra on the residuals (and, at least to me, it seems like I could even have ARMA-like spectra). The "classical" Lomb-Scargle periodogram, on the other hand, assumes white noise in order to test the significance of the peaks, so the significance test on the peaks doesn't seem like a good idea in order to test if there are any residual frequencies on the periodogram.

I've been trying to derive significance tests assuming ARMA-like noise, but the problem of defining the order $p,q$ of the process arises: do you know of a way of differencing between correlated noise and sine wave-like spectra? This paper shows a way of doing this assuming AR(1) noise (in fact something more like a CAR-type noise?), but the problem of it is that it actually knows there is red noise. I think I can "see" how to do it: sine wave-like spectra has almost-symmetric decreasing peaks around the main peak because of the (uneven) sampling, which ARMA-like noise doesn't have (the amplitudes are random with an apparent upper limit dictated by the form of the spectra). I've also been thinking in performing fits with ARMA and sine-like waves, and perform model selection via AIC...any suggestions?

share|improve this question
    
it is interesting question for me –  dato datuashvili May 11 at 17:33
add comment

1 Answer 1

A general test for periodicity is Fisher's periodogram test. I assume you are familar with it. But in case you aren't here are some links to recent papers related to it. http://users.ics.aalto.fi/harrila/publications/RobFisherPeriodicity.pdf

http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.aos/1247836665 This one is related but the Fisher in this article is Nick Fisher http://www.ifc.unam.mx/pages/curso_ritmos/capitulo4/Diggle_Fisher_1991.pdf

share|improve this answer
1  
I see two problems with Fisher's test: it assumes i.i.d. and even sampling (which I don't have). The only thing for what I can assume even sampling is for the modelling of the ARMA model for which I really don't need the physical description (where it would be better to use the continuous CAR and CARMA models), just the detection of it. –  Néstor Jun 29 '12 at 0:09
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.