I am doing my master's thesis and I must compare various forecasting techniques at different frequencies of datasets. I am using my universities dataset, the REDD dataset, UCI dataset and CER Ireland dataset for this purpose. The data I use is in seconds for a time span of a month and this gives > 3 million records.
I have been trying to understand how to make good use of all this data but couldn't exactly get to a solution. I have read Time series modeling with high-frequency data, but I don't understand and couldn't find resources how to apply it to my problem. I have tried reading several blogs and books to get an understanding of time series forecasting but most literature has examples with granularity only as low as hourly data. Some references I found were about high granularity data but only for a short period of time.
I have read in Prof. Rob Hyndman's blog that practically the ARIMA model can only calculate till 200 autoregressive points and if my understanding is correct then for data with a frequency in seconds, I could achieve daily trends only with $3600*24 = 86400$ previous values?
I am not sure how I should deal with this.
Here is how the data look (the y-axis is watts):