A neural network is a several linear transormations $L_1,\ldots, L_m$ that are sequentialy appilied to feature vector $X$. A compositon of linear transformations is a linear transformation. So after all we get $L X$ where $L$ is a composition of $L_1,\ldots, L_m$.

The question is: if eventually we have that neural network is just applying a liner transformation to a feature vector what is the essential difference betwen neural networks and linear regression


No, a neural network is not several consecutive linear transformations. As you note, that would only result in another linear transformation in the end, so why do many instead of one? Actually, a neural network performs several (at least one, but possibly more, depending on the number of hidden layers) nonlinear (e.g. sigmoid) transformations.

That is also the difference between a neural network and a linear regression, since the latter uses a linear combination of regressors to approximate the regressand.

  • $\begingroup$ (minor detail, which I guess you aware of but just for neophytes: A neural network is not necessarily several consecutive linear transformations.) $\endgroup$ – Franck Dernoncourt Feb 4 '17 at 15:36
  • $\begingroup$ @FranckDernoncourt, thank you. Yes, a network may have $m\geq 1$ layers, so it is of course possible that $m=1$. $\endgroup$ – Richard Hardy Feb 4 '17 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.