I am training a neural network to play tic-tac-toe. The input is the board (a vector of 9 doubles) and the output is a vector of Q-values (9 doubles).
The input vector is analyzed as follows: If a cell in the input vector is below -0.5 the cell is occupied by the Nought. If the cell is over 0.5 it is occupied by Cross and otherwise the cell is empty.
My goal is to create an AI that can play against a human as a Nought or as a Cross. I see several ways of trying to achieve this:
We are going to train only one model. The AI does not know if it is a Nought or a Cross. The training is executed as follows: The AI is going to play against itself by first picking an action according to the output of the current model. To get the reward for the action we have to predict how the opponent would respond to our action. We use our current model and pick an action to our opponent. After the opponent action is chosen we can calculate the reward and fit the model.
Same as above but instead of using the new state (state after our action) as such, we invert the new state (Noughts become Crosses and vice versa) before predicting the opponents action.
We are going to train two models. The AI knows which side it is playing and can use the specific model trained to play that side. The training is executed as follows: The Model for the Cross gets to start and pick an action. The action is appended to the state which is passed to the Noughts model so it can decide on an action. After Nougths model has chosen an action the reward can be calculated for Cross' model and the model can be fit against the new learned weights. Nought's model can be fit similarly after it has learned how Cross responded to its action.
Same as 1 but we fit the model for opponent moves too.
Same as 2 but we fit the model for opponent moves too.
Strategy 1 seems hopeless to me because the training happens from one side only and because of this I suspect that the chosen actions for the opponent are garbage.
I see strategy 2 as a little bit better than strategy 1 because the opponent moves are chosen from the inverted board, but this has the problem that the board states are never same as the states which the model was fitted for. (There is never a case where Cross would choose an action on a board having more noughts than crosses and be fitted for the chosen action. However the opponent actions are predicted on states like this)
Only minus I can think of with the strategy 3 is that AI has to know which side it is playing on to know which model to use to choose it's actions.
Strategies 4 and 5 seem like best choices (I don't know if one is better than other) and now we come to the question in the title. Is it possible for the AI to use only one model to play both sides or should I use two models? In addition: Which of the strategies would be the best choice? Or is there a strategy I have not thought of?