# What is the mathematical underpinning of feedforward artificial neural network?

For a school project, I have implemented a 3 layer feedforward ANN with an RBF activation function that can be used to distinguish between different types of signals.

I have a demonstration coming up soon and I am afraid that I will not be able to handle even trivial questions about ANNs from the other student audience such as "why does ANN work".

A lot of people assumes that there exists some sort of underlying mathematical principle. For example, in SVM we have this construction of a hyperplane that separates different features. In HMM we have this markov model.

But for ANN...I am truly unsure what mathematical model we are talking about here. The selection of layers, neurons, learning rate or even activation function seems a bit arbitrary.

Through some search I have found that it is related to gradient descend algorithm where we are trying to find a set of weights that minimizes the potential energy function. But how can I describe this in a simple way that people can understand especially when I do not understand it too good myself?

It depends in the application, in general NN work because it has been shown that given the activation functions, as the number of hidden layers $N \rightarrow \infty$ a single layer feedforward network can arbitrarily approximate all functions $f$ (this is known as the Universal Approximation Theorem https://en.wikipedia.org/wiki/Universal_approximation_theorem), this has been proven for both feedforward MLPs [Hornik1991] and RBFNs [i've forgotten the reference]. For classification FF MLPs essentially work in the same manner as an SVM, and generate a convex hull by the functions approximated by the network.