# What makes a neural network linear? [duplicate]

Two-part question: Neural Networks(NN) can be looked at as stacked units of logistic regression classifiers (LRC). A basic requirement of an activation function is to be non-linear. When LRC is a neuron, sigmoid function is the activation function and is said to bring non-linearity to NN.

Q1:) If this is true, why is LRC still a linear classifier even when it uses non-linear sigmoid function?

Further, if the activation function is a linear function(e.g. an identity function), the NN can no longer learn non-linear decision boundaries.

Q2:) Does this mean that the depth of a neural network plays no role in making it non-linear?

I have scanned similar threads for this question. But not convinced yet!

• What's the definition of a linear function? How can you apply that definition to the particular case of the neural networks you're considering?
– Sycorax
Dec 8, 2020 at 17:42
• Hi, thanks, I edited my answer a bit. A linear function's graph would have a straight line, it's output can be expressed as a sum of it's inputs (could be scaled). I say if we use the identity function as an activation function, the final output for an input 'x' can be expressed as Wx + B, where W encompasses matrix products of all Ws of all layers of NN. Same goes for B. In my opinion, that does make an NN linear when a linear activation function is used. Dec 8, 2020 at 17:58
• Yes, the composition of several linear operations is itself linear. Some more detail can be found here stats.stackexchange.com/questions/228296/…
– Sycorax
Dec 8, 2020 at 18:02
• Thanks. Is it okay to conclude that the non-linearity of a NN arises due to the combination of its depth and non-linear activation functions? Hence, either of the two alone will not suffice. Dec 8, 2020 at 18:16
• @Sycorax Ah! I see. Such overloadings have been a constant source of paradoxical intuitions for me. Thanks a lot for clarifying that they exist outside my mind too. Dec 8, 2020 at 18:27