I wonder what is the smallest amount of bits a neuron needs?
For example for speech recognition / image recognition tasks?
Can 4 bits produce as good results as 16 bits, or even 64 bits ?
Is it not a waste to use 16 bits ( or 64 bit double-s)?
For example spin systems can be encoded with only 1 bit per spin and there is a similarity between spins and neurons (Hopfield network).