# Why is number of neurons in hidden layers a power of two?

There is a statement in this quora answer:

Layer depth is usually a power of 2 because it is convenient for the GPU.


Also, in fully connected layers number of neurons in every hidden layer corresponds to a power of 2.

But, why is the power of 2 convenient for GPU? If I have everything for the power of 3, why would it be inconvenient for GPU? I thought one would use GPU in deep learning only because one can parallelize batches/or convolution computation.

• Is your computer a binary computer or a ternary one? – whuber Dec 7 '18 at 23:05
• How do I understand it? Each GPU will have 2 or 3 cores? – Alina Dec 7 '18 at 23:19
• It's more basic than that: ultimately, data are represented on your computer in a unit like a bit. How many states do the bits of your computer have? Two? Three? Ten? – whuber Dec 8 '18 at 14:58