Logistic Regression Vs Neural Networks This is what I have understood about neural networks. 
There are $L$ layers from input to output (including both) and the equations as follows.
$a(2) = \theta(1) x$
$a(3) = \theta(2)a(2)$
.
.
.
$y = a(L) = \theta(n-1) a(n-1)$
So, we can write $a(L)$ as below
$y = a(L) = \theta(1)\theta(2)\cdots\theta(n-1)x$
Let's suppose $\theta(1)\theta(2)\cdots\theta(n-1) = \Omega(L)$
Now we have $y = \Omega(L)x$
To solve the above equation, we can even have logistic regression techniques, which will be much easier.
So, what is the point in doing it so complicated, when we are not including any higher order terms?
Someone please explain
 A: It's generally true that classification problems can also be solved by regression.
Problems solvable by ANNs can also be solved by linear regression.  Problems solvable by decision trees can be also be solved by logistic regression.  Etc.
Abstractly, they do the same thing: prediction.
As to why people use one and not the other, the short answer is: it depends.  Sometimes on what the people implementing the solution are comfortable with, sometimes on what's available, institutional knowledge, etc.
The slightly longer answer to whether a specific regression model will perform better than a specific ANN or DBN on a specific dataset is: it still depends.  
There are numerous papers that have novel linear regression based solutions for, say, image recognition.  You can pit any one of these against the NN-based solution that comes with OpenCV and get wildly varying results.
You can replicate the inconsistency by picking the best regression model and then compare it to various and sundry classifiers.
Your results are based more on your familiarity with the algorithms, tools and data in question.  If there were a clearly, obviously, undeniably better solution, we'd be using it.  (Some places more quickly than others.)
A: Increasing the number of dependent variables is a problem for any prediction algorithm.  This includes regression models, NNs, SVMs, etc.
http://en.wikipedia.org/wiki/Curse_of_dimensionality
Huge input vectors are harder to classify the larger they get.  NNs are not immune to this.  However, NNs are advantageous in that by design they're networks.  They lend themselves to distributed environments well.  If your problem can be subdivided and executed simultaneously on a large number of compute nodes then an ANN optimized for running in a distributed environment is certainly an attractive option.
Parallelization of regression models is an ongoing area of research.
http://www.bing.com/search?q=logistic+regression+parallelization&pc=MOZI&form=MOZSBR
It's not something I'm deeply familiar with.  Apache Mahout apparently allows you to run the same regression model with different parameters simultaneously, but with a very large model (the ubiquitous millions of input vectors all with thousands of independent variables) you're still limited in how you can break each model down and take advantage of modern distributed computing options.
