1
$\begingroup$

I am trying to design a 1-hidden-layer neural network to implement parity bit checker for 5-bit length inputs, wherein each neuron has a simple threshold activation i.e.

$$ a(z)= \begin{cases} 1 & \text{if } z\ge threshold \\ 0 & \text{if } otherwise \\ \end{cases} $$ The parity checker should yield 1 for even number of ON bits, and zero otherwise.

The answer, I referred to, has $2^{(5-1)}=16$ neurons for the hidden layer. Whereas, I could design a network (see below) using only 5 neurons for the same layer. Wondering, what does this $2^{(n-1)}$ signify? Is it the max bound of hidden neurons for a 1-hidden-layer neural network? What is the minimum bound expression, if any?

Note: I know the concept behind the $2^{(n-1)}$, as in, how any binary function can be expressed as DNF (disjunctive normal form) of $2^n$ terms, and how they can be simplified to $\frac{2^n}{2}$ terms where each term can be implemented with a single neuron with threshold activation.

Note: I also found that my design with $5 (=\log_2{2^{n=5}} \div \log_2{2^{m=1}})$ hidden neurons matches with this paper. Is the lowest bound expression from this paper accepted widely!

enter image description here

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.