1
$\begingroup$

I wonder what is the smallest amount of bits a neuron needs?

For example for speech recognition / image recognition tasks?

Can 4 bits produce as good results as 16 bits, or even 64 bits ?

Is it not a waste to use 16 bits ( or 64 bit double-s)?

For example spin systems can be encoded with only 1 bit per spin and there is a similarity between spins and neurons (Hopfield network).

$\endgroup$

1 Answer 1

4
$\begingroup$

The following papers have studied this question (descending chronological order):

Example from Deep Learning with Limited Numerical Precision:

enter image description here


FYI:

$\endgroup$
2
  • $\begingroup$ These look like some great references, but I'd recommend adding a few words of summary to your answer, even if they're just quotes. $\endgroup$ Aug 27, 2016 at 17:18
  • $\begingroup$ Fantastic, thank you very much for your effort summarising the relevant research. $\endgroup$
    – user70990
    Aug 28, 2016 at 9:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy