1
$\begingroup$

I am writing an artificial intelligence of a game. The algorithm needs a neural network to represent some information of the game board. The board is an 8*8 grid and each position in the grid has 4 types of contents. Since the type of a position is a discrete value, I binarize the type to a vector of size 4 (according to Arun Iyer's suggestions). Thus, I can map the game board to a 8*8*4 tensor that feeds to the neural network.

Since the game board can be viewed (roughly) as an 8*8 image with 4 channels, I think a convolutional neural network may be helpful. Am I right?

If that's right, I am wondering if my design of the three convolution layers (all with stride 1) makes sense: a 3*3 kernel with 32 feature maps, followed by a 3*3 kernel with 64 feature maps and followed by a 2*2 kernel with 64 feature maps.

Through testing, I found this structure is helpful and I want to add more layers or more parameters. However, since the input is too small (8*8), adding more convolution layers will make the size of the last feature maps (before FC layers) very small (may be 1*1 or 2*2). Does small feature map worsen the model's performance? Or should I use some other network architectures?

Thanks in advance!

P.S. I am applying deep reinforcement learning on the game. The output of the network is the estimated rewards the agent could get under current game board configuration. The network is trained using q-learning.

$\endgroup$
1
$\begingroup$

As your feature maps get smaller, each of them is representing information about larger portion of the game board. Features 1x1 contain global information about the game board, whereas larger one contain more fine-resolution information (about individual parts of the board). It is not so clear what you are doing with the network output or how you train the network, so it is hard to say what is better in your case.

My guess is that small feature maps should be okay unless you need the network output to contain spatial information (which you lose by using a fully connected layer anyway).

$\endgroup$
4
  • $\begingroup$ I am applying deep reinforcement learning on the game. The output of the network is the estimated rewards the agent could get under current game board contents. The network is trained using q-learning. $\endgroup$ – madnerd Feb 4 '18 at 12:47
  • $\begingroup$ Then perhaps add these details into the question text $\endgroup$ – Jan Kukacka Feb 4 '18 at 12:48
  • $\begingroup$ Thanks! What do you mean by saying "the network output to contain spatial information"? Any examples? $\endgroup$ – madnerd Feb 5 '18 at 1:51
  • $\begingroup$ In the sense that if you have a feature map extracted by convolutional layers, each of the activations in the feature map corresponds to a particular location in the image. E.g. imagine a feature map 2x2. than the top left cell corresponds (roughly speaking) to the top left quarter of the input. $\endgroup$ – Jan Kukacka Feb 5 '18 at 9:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.