I've developed a model that performs electrical load forecasting by taking as input the following, in 1 hour resolution:
- Future 24 hours of temperature
- Historical load from 1 year ago with best matching temperature
- Historical load from 2 years ago with best matching temperature
- Historical load from 3 years ago with best matching temperature
- Historical load from 4 years ago with best matching temperature
The output is the predicted future 24 hours of electrical load, in 1 hour resolution. This is using an LSTM sequence to sequence model, though I don't believe the model matters.
I have 10 years of data, split so that the train and test data are 5 years each. Because I'm using historical data, the 'historical load' inputs will sometimes be from a future year if the time being forecast is early in the period of available data. For example, if the train dataset is from 01-jan-2008 to 01-jan-2013 and a forecast is being performed at 01-june-2010 then the historical loads will be from 2008, 2009, 2011, and 2012.
Now, when this model is used in production I want it to be able to pull the four historical days from as large a pool as possible. So instead of picking historical load from each of the past four years, it will pick the best historical load from each of the past ten or so years, then pick the best four out of those.
- Is it valid for the model to use historical data from the training set when running in inference mode?
- What about doing the same when evaluating the model on the test dataset?
I can imagine that by using the same inputs in the test and train situations the network may have over-fit those inputs and will produce an output that it has somewhat memorized. But, I'm not quite sure if this is a big issue or if there's anything else I haven't thought of - hence my qustions :)