I was reading this blog about Deep Q-Learning.
1- In the The input section of the blog, I wanted to know how do we feed the 4 still-frames/screenshots from the game, that represent the input state, into the Policy network? Will all 4 frames be fed in one flattened tensor (where one image ends the next one starts, forming a continuous row input, in one tensor)? Or will they be fed separately one after the other into the network?
2- For preprocessing the images, do we avoid using the Max-pooling stage? My understanding is this process eliminate the need for spacial/position recognition in image-feature recognition. While in normal Conv-Net this is important for recognising image features regardless of where they appear in space and distance (so we us max-pooling). In Q-learning for games, the space/position of different elements on the image is important. Therefore, we remove the use of Max-pooling from the proprocessing stage. Is this correct?
3- Can anyone recommend a good implementation resource of Deep Q-learning, written from scratch (in Python), i.e. without the use of out-of-the-box libraries, like PyTorch, Keras and Scikit-learn ..etc, for a game, where image frame feeds from the game is required as states input. I'm thinking perhaps implementing the model from scratch gives a better control over customisation and fine tuning of the hyper-parameters. Or is it better to use out-out-of-the-box library? Any code implementation on this would be super helpful.
Many thanks in advance.