The math component would likely include advanced algebra, trig, linear algebra, and calculus at minimum.
But also think outside the box. Good programming skills are also necessary including solid foundations in algorithms (Coursera has two courses on algorithms) and proficiency with MatLab, Octave, or R (and with a flexible programming language like Java, C/C++, or Python). I mention these in response to your question because they are more "applied math" skills in my opinion--and are fundamental to translating between theory and applied implementations.
I have taken a number of the Coursera courses related to machine learning (and agree with one other poster that Prof. Ng's Machine Learning is fantastic) and NN. A few months ago, Coursera hosted a Neural Networks Course (not sure if this is still available) through the University of Toronto and Geoffrey Hinton. A great course and demanded: knowledge of calculus, proficiency with Octave (an open source MatLab-like clone), good algorithmic design (for scalability), and linear algebra.
You might also (while not math per se), think about topics such as natural language processing (for feature extraction, etc.), information retrieval, statistics/probability theory as well as other areas of Machine Learning (to get more theory). Recent texts such as Foundations of Machine Learning (Mohri) or Introduction to Machine Learning (Alpaydin) might be helpful to you in bridging the theory-to-implementation complexity (just in my opinion, this can be a hard leap)--and both texts are very math heavy, especially Foundations.
Again, I think all relate to math and NN but in a broader sense.