I am working on training a reinforcement learning agent with a huge dataset of past human players' experience. Each user had to independently increase their game score. I am using the dataset because I have plenty of data and because on-policy training can be slow due to the game speed.
The only problem I have is the human player ID as a categorical feature. I have around 1000 experience play from each user and an ID should be assigned so the LSTMs can detect the sequential behavior of the game play.
1- Serve each player's experience as an episode to the agent so to get rid of user IDs as a category
2- or keep the training batches together including all the user IDs and train the agenet with them?
The problem here is that it is complicated for me to use one-hot encode for this large category.