Tic Tac Toe Neural Network Predicting a Piece Where there is a Piece

Question

I am creating a neural network to play tic tac toe. I have it play 1000 random games to gather the training data and append the winning data into a list of the board before the move as the X value and the move as the Y value. I am training the data with this line:

model.fit({'input': x}, {'targets': y}, n_epoch=50, snapshot_step=500, show_metric=True, run_id='openai_learning')

And the model that I am using is this:

network = input_data(shape=(None, 9), name='input')

network = fully_connected(network, 128, activation='relu')
network = dropout(network, 0.8)

network = fully_connected(network, 256, activation='relu')
network = dropout(network, 0.8)

network = fully_connected(network, 512, activation='relu')
network = dropout(network, 0.8)

network = fully_connected(network, 256, activation='relu')
network = dropout(network, 0.8)

network = fully_connected(network, 128, activation='relu')
network = dropout(network, 0.8)

network = fully_connected(network, 1, activation='linear')
network = regression(network, optimizer='adam', learning_rate=0.01, loss='mean_square', name='targets')

model = tflearn.DNN(network, tensorboard_dir='log')

But when I go to predict the next move it keeps on predicting to play the piece on the board where there is another piece. What would be the best way to prevent it? The only things that I can think of is finding the closest open piece on the board or tweak the parameters of training.

Here is the source file for the game:

https://raw.githubusercontent.com/ben60501/special-potato/master/TicTacToe.py

Answer 1

I have created neural networks to play tic-tac-toe before. Instead of having just 1 output, you should have 9 outputs. Each output representing the probability that one of the 9 squares on the board is a good move. This turns the game more into a classification problem.

Example (playing as O):

Given board:

X - X
O - O
X - -

The output could be:

0.1 0.5 0.2
0.7 0.6 0.1
0.3 0.4 0.4

Filter out the taken spots:

- 0.5 - 
- 0.6 - 
- 0.4 0.4

So your next move is the spot with p=0.6!

This is far better than chosing a random near spot when the outputted spot is already taken. While training, you have to make outputs look like this:

0 0 0 
0 1 0 
0 0 0 

So there must only be one 1 in the output array!