I am trying to design a 1-hidden-layer neural network to implement parity bit checker for 5-bit length inputs, wherein each neuron has a simple threshold activation i.e.

$$ a(z)= \begin{cases} 1 & \text{if } z\ge threshold \\ 0 & \text{if } otherwise \\ \end{cases} $$ The parity checker should yield 1 for even number of ON bits, and zero otherwise.

The answer, I referred to, has $2^{(5-1)}=16$ neurons for the hidden layer. Whereas, I could design a network (see below) using only 5 neurons for the same layer. Wondering, what does this $2^{(n-1)}$ signify? Is it the max bound of hidden neurons for a 1-hidden-layer neural network? What is the minimum bound expression, if any?

Note: I know the concept behind the $2^{(n-1)}$, as in, how any binary function can be expressed as DNF (disjunctive normal form) of $2^n$ terms, and how they can be simplified to $\frac{2^n}{2}$ terms where each term can be implemented with a single neuron with threshold activation.

Note: I also found that my design with $5 (=\log_2{2^{n=5}} \div \log_2{2^{m=1}})$ hidden neurons matches with this paper. Is the lowest bound expression from this paper accepted widely!

enter image description here



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.