How to train LSTM model on multiple time series data?
Use case: I have weekly sales of 20,000 agents for last 5 years. Need to forecast upcoming weekly sales for each agent.
Do I need to follow a batch processing technique - take one agent at a time, train LSTM model then forecast? Any better ways?
Make the identity of the agent one of the features, and train on all data. Probably train on a mini-batch of eg 128 agents at a time: run through the time-series from start to finish for those 128 agents, then select a new mini-batch of agents. For each mini-batch, run a slice of say 50 timesteps, then backprop. Keep the end states from that slice, and run the next 50 timesteps, starting from those end states. Rinse and repeat till you get to the end of the timesteps, for that mini-batch of ~128 agents.
By putting the identity of each agent as one of the features, you make it possible for the network to:
Edit: Alpo Jose wrote:
ok, do I need to use one-hot encoding to make identity of agent?
Ooo, that's true. There are 20,000 of them. That's kind of a lot. I think that what you might want to do is 'embed' them. Have a lookup layer, that takes in an agent id (expressed as an integer, an index), and outputs a high-dimensional vector, like something like a 50-300 length vector, probably a vector whose length approximately matches the hidden size of your LSTM.
Mathematically, a lookup table, also called an 'embedding layer', is equivalent to making the agent ids into one-hot vectors, then passing through a linear (fully-connected) layer. However, the requirements in memory are much reduced for the embedding layer.
In terms of what the embedding layer will learn, as you train it, the embedding layer will form some kind of latent representation of each agent. The latent representation won't likely be readable/interpretable in any way, but will allow the model to learn things like 'ok this agent, 1524, is relatively effective, but not on weekends; whereas 1526 is great every day; etc ....'. The latent dimensions of the embedding vector might actually mean stuff, but no-one ever tries to figure out what they mean (I think that would be hard/impossible). However, the high-dimensional per-agent embeddings allow the model to learn something about the behavior of each agent, and model this in the time-series predictions.
model = Sequential(); model.add(Dense(100, activation='relu', input_dim=X_train.shape[1])); model.add(Dense(1)); model.compile(loss='mse', optimizer='adam'); model.fit(X_train.values, Y_train, epochs=20) where X_train.shape = (1000, 30) and Y_train.shape = (1000,), i.e. each time series has 30 timesteps. In this case how can we use the agent id as a feature?
$\endgroup$
Commented
Mar 7, 2022 at 18:10