- The neural network with binary output with one or more hidden layers - No. Because Linear Regression = Input matrix * Weight Matrix = Output Score. Linear regression will have no hidden layers. When this output score is subject to a step up activation function or a threshold then we are getting into linear binary classification.
- Some site claims linear regression means the continuous value output. If I have an MLP with hidden layers, and its output is continuous value (ex: house price), then is it called linear regression? This is tricky and conceptualization is important. ANN can solve regression problem that is problems that incomewith continuous outcome variable. ANN is a flexible and complex algorithm. It can dynamically pick the best regression model, be it linear, logistic, or polynomial and if the prediction is not accurate enough it has hidden layers at its arsenal to boost the prediction power with higher accuracy. So, ANN can do the regression job. However, linear regression works best only when the linear regression equation is the best fit to the data available. This is the difference.
- Neural Network with linear activation functions ( doesn't matter binary output, continuous output value, hidden layer)
See, when you have linear activation function, it turns all layers into one as the linear combination of all layers with be a linear, thereby reducing it to an input output linear function which is a nothing but linear regression.
On a quick note, a single hidden layer with a sigmoid activation function and may be in combination with a step up function is a logistic regression.
Hope this answers all your doubts and confusion.
