5
$\begingroup$

Looking for some pointers on how best to structure a neural net that deals with a card game.

In Gin (rummy), you have a 10-card hand and you're trying to make melds and sets out of your cards. A meld is 3-5 cards of the same suit in sequence (i.e. 3♦,4♦,5♦), while a set is 3-4 cards of the same rank (i.e 3♦,3♣,3♠ or 6♣,6♥,6♦,6♠). Players take turns drawing from either the deck or the top of the discard pile, then choosing a card to discard. When they discard, if they have 10 or fewer deadwood (unmatched cards) they can discard face-down and declare a "knock". With exactly 0 deadwood, they can "knock gin" and receive a 25-point bonus for the hand.

My concern has to do with the AI discarding cards that are already paired up. I've seen it do this a number of times, and while I could be naive and say "it hasn't evolved those smarts yet," I wonder if it will be able to evolve those smarts at all.

I'd love some feedback as to how best to structure the input layer such that the weights form meaningful abstractions around things like sets and melds. My input layer is structured like so:

  • There is one Perceptron per card in the hand (so 11 total). The hand is unsorted and the sequence of cards may change at any time.
  • There is one Perceptron per card in the discard pile (so 30 total -- the game ends if only two cards are left in the draw pile: 10+10+30+2 = 52)
  • (unimplemented) There is one Perceptron per card for the opponent's hand. In the case that the opponent knocks improperly, their hand is played face-up.

I have one hidden layer, about the same size as the input, and an output layer like so [with options]:

  • First action of the hand [draw, pickup-from-discard-pile]
  • Second action of the hand [discard, knock, knock-gin]
  • Index of card for second action [0 through 10]
  • Accept an improper knock? [yes, no]

All that out, here's my concern: assume the player's hand is randomly shuffled after each move. How can the weights possibly balance for this? Presumably, the optimal weights for the input layer that deal with the player's hand will all be the same. Can the hidden layer accommodate this? Do I need a second or third hidden layer? I've considered reworking the player's hand to act as 52 Perceptrons, each of which is a yes/no answering the question, "Is card X in my hand?" My gut tells me not to do this, as if I'm giving the AI too much knowledge about the problem domain. I want the AI to remain as expert knowledge-free as possible.

$\endgroup$
  • 1
    $\begingroup$ Did you make any progress on your own? If so you can always answer your own question and share with the community. $\endgroup$ – Erik Sep 18 '15 at 17:45
2
$\begingroup$

I don't see this as either a classification or a regression problem. This is a reinforcement learning problem.

For neural networks you need to have sparse or continuous inputs. This means that your best bet is to encode card numbers and suits as binary vectors. Tehse will be very sparse e.g. there are cards from 1 to 14 and suits from 1 to 4. One vector will therefore have length 14 while the other 4. You thus encode one card with a vector of 14 + 4 = 18 binary values. This describes only one card.

If you are interested in neural networks and reinforcement learning (deep reinforcement learning) you might want to take a look at the work being done at deepmind.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I know this is super old, but I came across this question so I'm assuming plenty more people will in the future. If anyone wants resources on Reinforcement Learning (RL) and Deep RL, Sutton and Barto's Introduction to Reinforcement Learning is THE book on RL, I think they've been working on a new edition that discusses the more recent work done and drafts of that have been lurking around the internet. Berkeley's Deep Reinforcement Learning course taught by Sergey Levine is also really awesome too (really intense math btw since it's a graduate level course) if anyone wants to check that out. $\endgroup$ – vaughnmcbob1 Jan 17 '19 at 4:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.