I'm working on a bot that plays an imperfect information game similar to chess, where each move you are effectively moving a piece from one location to another. I'm trying to decide what the best way to format the input/output for my DQN, which I plan to use to determine what the best action is in a given state. However, I have been struggling to find any resources for using DRL on a problem like this.
The possibilities I see are the following:
passing the board into a convolutional layer and then once I reach a full layer, also passing in the action (which will be something like two coordinates) and having this return a q value.
passing the board into a convolution layer and having another dimension of the board be the coordinates for the action (so this is also passed into the convolution layer) and having this return a q value.
taking a greedy prediction of what the board will look like after my move (which ignores the opponents move and anything new discovered) and using that new greedily chosen state as input to the convolution layer, which will basically be a value function for a state.
Instead of a q value as the output, outputting a matrix the size of the board and attempt to teach it to indicate the best action through the board output.
It seems like DQN usually will either output a vector which is effectively the actions one hot encoded, but that doesn't seem incredibly applicable here. Especially since at any point there are many, many invalid actions and only a few valid actions. I'll inevitably try each until one works, but if anyone knows of any implementations of DRL for imperfect information games, that would be very useful in figuring out what works best off the bat.