I wonder what is the smallest amount of bits a neuron needs?

For example for speech recognition / image recognition tasks?

Can 4 bits produce as good results as 16 bits, or even 64 bits ?

Is it not a waste to use 16 bits ( or 64 bit double-s)?

For example spin systems can be encoded with only 1 bit per spin and there is a similarity between spins and neurons (Hopfield network).


1 Answer 1


The following papers have studied this question (descending chronological order):

Example from Deep Learning with Limited Numerical Precision:

enter image description here


  • $\begingroup$ These look like some great references, but I'd recommend adding a few words of summary to your answer, even if they're just quotes. $\endgroup$ Aug 27, 2016 at 17:18
  • $\begingroup$ Fantastic, thank you very much for your effort summarising the relevant research. $\endgroup$
    – user70990
    Aug 28, 2016 at 9:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy