0
$\begingroup$

Question 1. How to input multiple variables (features) x1, x2, x3...x10 which are in categorical in nature to neural network. Basically I want to know how will I prepare my input layer to neural network. Below is the sample example of my data set.

x1,  x2,        x10    Y
A,  Red         A10    1
B,  Blue        A20    2
C,  Green       A30    3

I have googled on this, some say use one-hot encoding. But I am not sure how will I apply one hot encoding technique to all the feature I have and how the input matrix structure will be.

If I have only one feature say x1. I understand I can apply one-hot encoding.

A:  1 0 0 
B:  0 1 0
C:  0 0 1

And this 3x3 matrix can be used as input layer to Neural network.

[3x3] -> [Neural Network] -> [out] 

But I have multiple feature here x1, x2 ... x10. How will I frame my input layer, what will the input layer structure. Please advise.

Question 2

Also, variable x2, x10 might get new categorical value which is not seen in training set (for eg. X10 = Z99 ) how to handle this problem.

$\endgroup$

marked as duplicate by Sycorax neural-networks Aug 1 at 15:52

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • $\begingroup$ Your question 2 has already been asked here but got no answer. Regarding question 1, what is it you find unclear? One-hot encoding is a standard thing, libraries like scikit-learn can do it $\endgroup$ – Jan Kukacka Aug 7 '18 at 9:50
  • $\begingroup$ @JanKukacka I understand I can apply One-hot encoding if have only one categorical variable say x1. But I have multiple features here, how will you apply one-hot encoding and what will be your input layer looks like. Can you please explain. $\endgroup$ – shakthydoss Aug 7 '18 at 10:19
3
$\begingroup$

@JanKukacka gave a good answer, but I just want show different solution that more efficient if you have high cardinality categorical features (features with large number of categories), called embedding. All categorical features can have one separate input or you can have different inputs per each categorical feature if you want to have embedding vectors with different shape. Each value from the category maps to some unique vector and at the end all these vectors and numerical features are concatenated into one large vector that gets propagated through the network.

One example you can find here: https://github.com/itdxer/neupy/blob/master/examples/mlp/mix_categorical_numerical_inputs.py

Here is how the graph looks like if you run this command

network.network.show()

You can see that input on the right side expects numerical identifiers per each categorical features. Three categorical features are transformed into three 4-dimensional vectors and after that they are just reshaped in order to form one vector per each sample rather than matrix. After that, vector with 12 values , that represents 3 categorical features (3 x 4) is concatenated with 17 numerical features. At this point you have 29 features in the vector and you can propagated it through the network.

$\endgroup$
  • $\begingroup$ A useful resource for understanding this: fast.ai/2018/04/29/categorical-embeddings in particular click the link after "The material from this post is covered in much more detail starting around 1:59:45" $\endgroup$ – Dan Aug 1 at 17:01
2
$\begingroup$

One-hot encoding converts a single categorical variable to several binary variable, out of which only one has True value and the rest is False, as:

Sample    |  Color               Sample |  Color=Red  | Color=Blue
--------------------            -----------------------------------
1         |  Red          =>     1      |  True       | False
2         |  Blue                2      |  False      | True

If you have several categorical variables, the same will happen to each of them. You will have several new groups of one-hot encoded variables:

Sample | Color | Size        Sample | Color=Red | Color=Blue | Size=Small | Size=Large
----------------------       ---------------------------------------------------------
1      | Red   | Small   =>  1      | True      | False      | True       | False
2      | Blue  | Large       2      | False     | True       | False      | True
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.