Best approach for energy demand forecasting

Question

I am trying to predict the amount of energy demand(Wh) of the next two weeks per hour. The dataset I have, contains each hour of each day since 2019 of the energy demand, is something like this:

X = [[[day],[month],[year],[hour],[consumption]],
      [[05], [01],  [2018],[10],  [150000]],
      [[05], [01,   [2018],[11],  [153000]],
      ...till today]

The thing is that I have fed this dataset (after preprocessing) to a MLP(dense fully connected layers) and it seem to give good results in the validation set, but this validation set is formed by a sample of the X dataset, so it is likely that the training set has the information of the demand of the hour before and after of every one in the validation set, which would be something like cheating since from one hour to next one there are little changes in demand, and because the real case is that I would like to have a model which can predict one/two weeks further in time the energy demand per hour. I have thought that maybe there is a kind of way to fit a model only using, when predicting a given hour' demand, the subset of the training set that is two weeks or more before on time that the given hour. This is what I have thought but I don't know if it is possible or if there are other approaches better than this one. What do you think?

Answer 1

You should split the time series instead of using a subsample as a test set, to avoid the problem you mentioned. You can find more here: https://stats.stackexchange.com/users/205266/wind.

Also regarding your problem, I think the energy demand has cyclic features. I suggest you take a look at the book "Introduction to Machine Learning with Python" by Andreas C. Müller and Sarah Guido (ed. O'Reilly). Look for Figure 4-12 "Number of bike rentals over time for a selected Citi Bike station" and the text there, I think they solve in a nice way the very same problem you are trying to solve.

EDIT: Following the comments, I add this update.

First, I don't think this is a general problem, so MLP is not too relevant in the discussion actually. Also, I assume here that you are familiar with cross-validation. What I understand about the procedure depicted in that graph is this. A way to do train the model is to keep let's say the last 3 months as a test set, and the previous data as training (after the split, the last part of the data always as test, of course, since you want to predict the future knowing the past). However, only with that you would compute a single value for you model accuracy. To do it better, you can do cross-validation in which case you split the same data in a different ways, with new training and test sets. That would give you different values for the accuracy. Proceeding in this way, you can average many values of accuracy to get a better, more robust and significant estimate of how you model performs, for the set of parameters you use (I assume you model has some free parameters that you can tune).

What is specific about time series is that the partition between training and test set only makes sense if the test set is, in time, after the training data, since as I said before you want to predict the future based on the past. Thus, if you have, for the sake of the argument, some time index t=1,2,3,.....100 you can use first a training set t=1-90 and a test t=91-100. Then, you repeat, this time with a training set t=1-80 and a test t=81-90 etc. until training t=1-10 and test t=11-19.

Regarding this question "Because that is too much data for training in comparison with the data you are predictiong": if your problem is that you cannot afford to do cross-validation, ok it's not mandatory, it's just desirable. However, if your problem is that you have so many training data that you cannot possibly train the model due to computational limitations, then you can indeed sub-sample your data to reduce it. For example, make time bins an average the values in each bin. Or simply remove 1h from every 2h (just try not to systematically remove the most meaningful hours! This is something you can also play with, since some sub-samplings can work while others can ruin the model performance). Then, from the sub-sampled data, you can create the test set as described above. Also, you should perform the same sub-sampling operation on future data you are given before making a prediction (well, you can check how your model behaves, but probably you will have to).