0
$\begingroup$

I have two datasets of daily irradiance measurements from a single site over a multiple year period. One takes readings every 15 minutes (A) and the other every 30 minutes (B). I trust the B dataset, and I want to test the accuracy of the A dataset compared to B.

Preferably, I want to use an interpolation function that fills in the 15 minute gaps between the B dataset. If that's not possible, I can also delete every other data entry in the A so that it will match the timestamps and be the same length as B.

Then I would like to use some statistical test to determine whether A is similar enough to B and accurate, or whether it is full of errors and trash.

The problems I see are that data for irradiance fluctuates over the seasons (i.e. more intense/higher irradiance values in summer while less in winter). Also the irradiance data goes to zero from ~6:30 pm to ~6:30 am the next day (sun doesn't shine at night, unless you're in Alaska lol ). So over half the values in my datasets are zero.

What's the best way to see if A is similar to B? I was thinking that I could, for example, pull all the irradiance values for 12pm each day, which should give me somewhat of a normal distribution, then perform a paired sample t-test on it. Then similarly do this for every hour/half hour of the day.

What are your thoughts? Thanks!!!

$\endgroup$
1
$\begingroup$

Have a look at ARIMA models to see if you can decompose each time series into seasonality (fluctuates over summer vs. winter) and trend components (year over year changes). As these measures should be independent of the sampling frequency (15 mins versus 30 mins), you can then compare the measures for A and B to determine if there is a relation between A and B.

Just to confirm that the sampling period doesn't afect the analysis I would also average successive pairs of A into 30 minute samples and then perform ARIMA on equal-sized time series.

You do in fact have two seasonalities, in that sunlight varies both across the day and across the year. Another approach would be to calculate the average per day for both A and B and perform an ARIMA decomposition on the daily time series, which of course should be directly comparable.

For performing ARIMA the auto.arima() function in Rob Hyndman's R package forecast is very powerful:

https://www.otexts.org/fpp/8/7

If you really want to compare both the daily and yearly seasonality, check out Hyndman's TBATS() function which supports multiple seasonality decomposition:

http://robjhyndman.com/hyndsight/forecasting-weekly-data/

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.