A neural network is a several linear transormations $L_1,\ldots, L_m$ that are sequentialy appilied to feature vector $X$. A compositon of linear transformations is a linear transformation. So after all we get $L X$ where $L$ is a composition of $L_1,\ldots, L_m$.

The question is: if eventually we have that neural network is just applying a liner transformation to a feature vector what is the essential difference betwen neural networks and linear regression


1 Answer 1


No, a neural network is not several consecutive linear transformations. As you note, that would only result in another linear transformation in the end, so why do many instead of one? Actually, a neural network performs several (at least one, but possibly more, depending on the number of hidden layers) nonlinear (e.g. sigmoid) transformations.

That is also the difference between a neural network and a linear regression, since the latter uses a linear combination of regressors to approximate the regressand.

  • $\begingroup$ (minor detail, which I guess you aware of but just for neophytes: A neural network is not necessarily several consecutive linear transformations.) $\endgroup$ Commented Feb 4, 2017 at 15:36
  • $\begingroup$ @FranckDernoncourt, thank you. Yes, a network may have $m\geq 1$ layers, so it is of course possible that $m=1$. $\endgroup$ Commented Feb 4, 2017 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.