Everywhere in the theory of neural networks, authors saying that idea came about by observing the work of the human brain. But I can not believe in that. I guess, everything is much simpler and neural networks is specific functions of math's series. Proof of this is the existence of the Weierstrass theorem and Taylor series, which says that every function can be approximated by certain polynomials. Am I right?
Neural network idea can be very loosely explained as following:
a. Using enough hidden layers can lead to the representation of every function that exists (again, this is loosely and is far from practical)
b. Since now you can represent every function in the world with parameters (weights) now use stochastic gradient decent on the weights (this is the back propagation in super short) to get to the optimal classifier (or regressor)