I have just covered Artificial Neural Networks on Coursera's Machine Learning course and I would like to know more theory behind them. I find the motivation that they mimic biology somewhat unsatisfactory.

On the surface it appears that at each level we replace covariates with a linear combination of them. By doing it repeatedly we allow for non-linear model fitting. This begs the question: why the neural networks are sometimes preferred to just fitting a non-linear model.

More generally, I would like to know how Artificial Neural Networks fit within the Bayesian Framework of inference which is described in detail in E.T. Jaynes' book "Probability Theory: The Logic Of Science". Or, to put it simply, why do artificial neural networks work when they work? And, of course, the fact that they make successful predictions implies that they follow the aforementioned framework. Please include relative references in your answers.

I have just covered Artificial Neural Networks on Coursera's Machine Learning course and I would like to know more theory behind them. I find the motivation that they mimic biology somewhat unsatisfactory.

On the surface it appears that at each level we replace covariates with a linear combination of them. By doing it repeatedly we allow for non-linear model fitting. This begs the question: why the neural networks are sometimes preferred to just fitting a non-linear model.

More generally, I would like to know how Artificial Neural Networks fit within the Bayesian Framework of inference which is described in detail in E.T. Jaynes' book "Probability Theory: The Logic Of Science". Or, to put it simply, why do artificial neural networks work when they work? And, of course, the fact that they make successful predictions implies that they follow the aforementioned framework.

I have just covered Artificial Neural Networks on Coursera's Machine Learning course and I would like to know more theory behind them. I find the motivation that they mimic biology somewhat unsatisfactory.

On the surface it appears that at each level we replace covariates with a linear combination of them. By doing it repeatedly we allow for non-linear model fitting. This begs the question: why the neural networks are sometimes preferred to just fitting a non-linear model.

More generally, I would like to know how Artificial Neural Networks fit within the Bayesian Framework of inference which is described in detail in E.T. Jaynes' book "Probability Theory: The Logic Of Science". Or, put simply, why do they work when they work? And, of course, the fact that they work implies that they follow the aforementioned framework. Please include relative references in your answers.

I have just covered Artificial Neural Networks on Coursera's Machine Learning course and I would like to know more theory behind them. I find the motivation that they mimic biology somewhat unsatisfactory.

On the surface it appears that at each level we replace covariates with a linear combination of them. By doing it repeatedly we allow for non-linear model fitting. This begs the question: why the neural networks are sometimes preferred to just fitting a non-linear model.

More generally, I would like to know how Artificial Neural Networks fit within the Bayesian Framework of inference which is described in detail in E.T. Jaynes' book "Probability Theory: The Logic Of Science". Or, to put it simply, why do artificial neural networks work when they work? And, of course, the fact that they make successful predictions implies that they follow the aforementioned framework. Please include relative references in your answers.