One year of historical data with yearly and daily seasonality

Question

I have 14 months(01/07/2018 to 30/08/2019) of one minute data, which I have aggregated to 10 mins block. So I have a data of dimension "61056 * 350". From this I am using 12 months of data to train the model and 2 months of data to validate it. I am using R version 3.6.0 to build my model.

Since my data is a data on temperature it clearly has multiple seasonality. One is each day and the other is by seasons. But since I have only one year data the model is not able to learn the yearly pattern of the data. Which I believe will surely impact the model. I am using df<-ts(train$DV,start = c(2018,7),frequency = 144). Is it correct else What can be done to deal with that? Or, should I use df<-msts(...,seasonal.periods = c(144,144*365)?

144*365 is the total number of observations for my training dataset.

Answer 1

You are right that not having enough weather data is a disadvantage - since you won't be able to establish the seasonality patterns from just one year.

In the absence of having enough data, one potential solution could be to simulate weather data using the same parameters as your current series.

As an example, let's assume the mean temperature in February from your dataset was 4°C, with a standard deviation of 5°C.

To simulate 30 days of data for February, one can do as follows:

> x=rnorm(30, mean=4, sd=5)
> x
 [1] -0.6768765  1.5813402  2.1578451  4.7855979 10.1800887
 [6]  5.9011200 -1.9421929  6.9911878  1.1797813 14.7528973
[11] -3.3971768 -0.6103933  1.6045041  0.4450205  3.8475489
[16] -1.0573087  4.1296465  2.2150627  8.5207687  2.4678347
[21] 14.1010963  9.5788337  5.4028398  7.1770535  2.0815703
[26] -0.3350638 -2.8182148 10.0825681  7.8359302  5.9970788

This gives a 30-day simulation of temperature fluctuations for that month.

Now, let's assume average July temperature was 35°C, with standard deviation of 5°C.

> x=rnorm(30, mean=35, sd=5)
> x
 [1] 31.59935 26.15818 29.61565 34.97618 41.51375 30.10105 35.74415
 [8] 34.32421 33.70380 35.78035 42.53513 42.62634 33.10639 42.99213
[15] 31.43488 38.51095 34.96452 27.68342 33.42387 44.38061 29.38172
[22] 33.26960 26.41184 30.83903 29.99817 43.94391 41.60079 30.08716
[29] 34.12541 38.34689

This example was in days, but you could also simulate for minutes, i.e. 43800 minutes in a month.

By generating a simulated dataset, creating a longer time series with this simulated data will allow you to gauge the seasonality trends more effectively.

Edit: As mentioned in the comment below, one solution if working with minute data is to generate simulations by minute instead of by day - which should solve the ordering issue.

> feb630am=rnorm(30, mean=4, sd=2)
> feb630am
 [1] 4.3503904 6.1305820 5.8328785 4.3971140 5.5445619 2.4979280 4.0735529 1.1415731 5.1418169 4.0068420 3.5821455
[12] 6.1484444 2.9203494 0.1494928 1.8483772 4.7708885 0.5346982 0.1477401 3.4053309 3.5267691 3.1811524 0.7689025
[23] 2.9950186 6.1302543 3.2114578 7.2501330 5.1915775 2.3814384 3.1654849 5.1760977
> 
> feb330pm=rnorm(30, mean=8, sd=2)
> feb330pm
 [1]  7.430943  7.872754  9.586560  5.546276  6.768111  8.837201  9.157584 10.499831  5.140073  6.030043  6.511989
[12]  5.877175  7.363821 10.534535 10.675319  6.905397  8.032596  8.807316  5.522470  4.937094  6.737235  4.774886
[23]  7.670951  7.563685  3.870916  8.267108  6.285726 10.083424  9.300405 10.280809

Each temperature reading generated here is for 6:30am and 3:30pm respectively, but on a different day. e.g. 4.35°C is the temperature at 6:30 am on 1st February, 6.13°C is the temperature at 6:30am on 2nd February, etc.

Then, it is a matter of sorting that data effectively so that all the temperature readings by minute are in sequence. As I said, this will involve quite a lot of "data wrangling", but is one solution if you wanted to analyse this by minute. Another solution could be to increase the time interval, e.g. every 1 hour, 4 hours, etc.