How are game difficulty levels programmed in TD gammon? I understand that Temporal Difference player is trained by making it play against itself. In such a case, how are game difficulty levels programmed?
Can fewer training iterations successfully create weaker players? Or just incompletely trained players?
 A: I think that using not fully trained agents for difficulty below grandmaster might be hard to do, because the designer of training process has to focus both on creating the "best" agent in the end AND setting the right cutoffs in between. Given the often stochastic nature of balancing exploration and exploitation during the learning (e.g. by using $\epsilon$-greedy policies), the intermediate agents might be not stable.
Instead I think it would be easier to first learn the state($s$)-action($a$)-value-function $Q(s,a)$ for the best agent and then try to adjust the parameters in such a way, that suboptimal actions are selected (by e.g. selecting an action with lesser $Q(s,a)$ or reducing the lookahead in heuristic search, resulting in next-best-action without taking long-term-effects into account). 
How to set the threshold for the different difficulty levels, i.e. what is "easy" etc., is a subject on its own (Game Design). The best agent defines the grandmaster level, but how many games should a beginner be able to win on average to define the lowest difficulty level ? 
Disclaimer 
I have only read the summary of TD-Gammon in the book Reinforcement Learning: An Introduction by Sutton and Barto, not the full paper.
A: In addition to selecting suboptimal actions as steffen mentioned, you can act on the following parameters when training TD-Gammon:


*

*number of games TD-Gammon was trained on

*number of units used in the hidden layer

*whether to use a 1-ply search, 2-ply search, 3-ply search, etc.

*which human-engineered feature to use


From http://videolectures.net/rldm2015_silver_reinforcement_learning/?q=reinforcement%20learning (David Silver):

The neural network that was used (image taken from http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node108.html): 

