# Binary Encoding vs One-hot Encoding

What is the difference between binary encoding and one-hot for categorical input variables for English Text and their impact on the neural network? Can anyone help me to find a scientific paper about this problem?

If you have a system with $$n$$ different (ordered) states, the binary encoding of a given state is simply it's $$\text{rank number} - 1$$ in binary format (e.g. for the $$k$$th state the binary $$k - 1$$). The one hot encoding of this $$k$$th state will be a vector/series of length $$n$$ with a single high bit (1) at the $$k$$th place, and all the other bits are low (0).

As an example encodings for the next system (levels of education):

Level "Decimal encoding" Binary encoding One hot encoding
No 0 000 000001
Primary 1 001 000010
Secondary 2 010 000100
BSc/BA 3 011 001000
MSc/MA 4 100 010000
PhD 5 101 100000

References: One hot encoding at Wikipedia

And a 2017 paper on the comparison on the effects of different encodings to neural networks in the International Journal of Computer Applications could be a good starting point: A Comparative Study of Categorical Variable Encoding Techniques for Neural Network Classifiers

• Thank you very much but for level of character Jan 27, 2018 at 9:30
• What do you mean by "but for level of character"? Sorry, I don't get it. Jan 27, 2018 at 10:51
• when make the pre-processing of data(encoding) to enter the model we represent each character to zeros and ones not word Jan 27, 2018 at 12:31
• Also, if you have $n$ unique categories (or words here), OHE results in either $n$ or $n-1$ features where as binary encoding results in only $\log_{2} n$. So if your vocabulary is $100$ words then OHE needs at least $99$ features whereas binary encoding needs only $7$ which is a major reduction in dimensionality.
– Dan
Feb 8, 2018 at 8:03
• I just read this paper: diva-portal.org/smash/get/diva2:1259073/FULLTEXT01.pdf Unsuprisingly, one-hot encoded variables give the best precision. I think it's a good practice to only use binary encoding when we are limited by resources.
– neeh
Feb 7 at 4:01