Two-part question: Neural Networks(NN) can be looked at as stacked units of logistic regression classifiers (LRC). A basic requirement of an activation function is to be non-linear. When LRC is a neuron, sigmoid function is the activation function and is said to bring non-linearity to NN.
Q1:) If this is true, why is LRC still a linear classifier even when it uses non-linear sigmoid function?
Further, if the activation function is a linear function(e.g. an identity function), the NN can no longer learn non-linear decision boundaries.
Q2:) Does this mean that the depth of a neural network plays no role in making it non-linear?
I have scanned similar threads for this question. But not convinced yet!