# Can a unit-step function introduce non-linearity if used in the hidden nodes?

I have a follow up from this question. From the answers in that question, it's obvious that at minimum, a shallow network (only has a single hidden layer), as defined in the "Universal Approximation Theorem," is needed to have any sort of non-linearity. The original perceptron only had input and output nodes and used the unit-step function, hence it was simply a linear classifier.

If I have a shallow network and I use the unit-step function as the activation for the hidden nodes and the output nodes, is the model considered a non-linear classifier? I realize that this activation function is not desirable as it is not continuous/differentiable. But regardless of desirability, will this model be considered non-linear?

In fact, to take it a step further, if I had a shallow network and I used unit-step activation for the hidden nodes, but a linear activation for the output, this would still be considered a model that can (no guarantee that the training algorithm can train the model) solve non-linearly separable problems, is that correct?

Thus we see that the function $$y(x_1,x_2)$$ can be expressed as a linear combination of step functions whose arguments are linear combinations of $$x_1$$ and $$x_2$$. In other words the function $$y(x_1,x_2)$$ can be approximated by a two-layer network with threshold hidden units and linear output units.