4
$\begingroup$

Suppose you have 2 distinct features x1 and x2 with discrete values between 1 and 10.

When building a neural network, I have 2 options:

  1. Create a new feature x3 = 10*x1 + x2 and use this as the only input.
  2. Use both inputs x1 and x2 for the neural network.

How can I decide which is the best option?

Here is a practical example inspired from poker: You have 7 cards. The first 2 cards are from player 1. The next 2 cards are from player 2 and the last 3 cards are the flop. You want to determine who has the strongest hand. The two options are:

  1. Treat each card as a number from 1 to 52.
  2. Make two inputs: One for the rank (2,3,4,5,6...J,Q,K,A) and one for the suit (club,diamond,heart,spade).

More precisely, I want to understand how choosing each option will affect how much training data is needed, how many neurons in the hidden layer I should use and how much time will be needed for training.

$\endgroup$
2
$\begingroup$

Often when you're dealing with categorical data, it's most useful to convert the elements of your dataset to use a "one-hot" code for your inputs. In the poker example that you gave, the cards in a deck certainly fall into this category of input, since the face value of the card might not map linearly onto the card's value for your hand.

Actually, the cards example is a very interesting one, because (at least from my naive perspective) there are many different features of each card that might be relevant to predicting a card's value. A naive one-hot encoding would use a vector of length 52 (assuming no jokers) for each card in the deck. The ace of spades might be represented as [1, 0, 0, ...] (for a total of 51 zeros and 1 one), the two of spades might be represented as [0, 1, 0, ...], and so forth.

However, each card has a suit that is orthogonal to its value, so you might be able to factor the encoding for each card into two one-hot vectors, concatenated together. The suit could be a vector of length 4, and the value of the card could be a vector of length 13. Then the ace of spades might be represented as [1, 0, 0, 0] + [1, 0, 0, ...] (for a total of 15 zeros and 2 ones), the two of spades might be [1, 0, 0, 0] + [0, 1, 0, ...], and so on.

If you want to use such an encoding as an input for your neural network, then you'd need many different input units. For example, if you wanted to train your neural network to map two cards onto some output value, then your network would need either 34 inputs (if you use the factored encoding) or 104 inputs (if you used a naive one-hot encoding).

Certainly a disadvantage of such an input encoding is that your training data will be larger, but the encoding itself might identify valuable information about the inputs that might otherwise be difficult for your network to locate.

$\endgroup$
0
$\begingroup$

well your aim in choosing inputs is really to make the problem as linear as possible. so to answer your poker question... a) fix 6 cards now can you draw the "points curve" for the "number version (1-52)" i'm not so good at poker, but I guess its pretty nonlinear whereas the two inputs one should be much simpler

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.