11
$\begingroup$

Not sure if this is appropriate for this site, but I'm beginning my MSE in computer science (BS in applied mathematics) and want to get a strong background in machine learning (I'm most likely going to pursue a PhD). One of my sub-interests is neural networks.

What is a good mathematical background for ANNs? Like in other areas of machine learning, I assume linear algebra is important, but what other areas of mathematics are important?

I plan to read Neural Networks: A Systematic Introduction or Neural Networks for Pattern Recognition. Does anyone have any input or alternative recommendations?

Improve this question
$\endgroup$
0
10
$\begingroup$

The second reference you give is, in my opinion, still the best book on NN, even though it might be a bit outdated and does not deal with more recent developments like deep architectures. You will get the basics right, and become familiar with all the basic concepts around machine learning.

If you go through the book, you will need linear algebra, multivariate calculus and basic notions of statistics (conditional probabilities, bayes theorem and be familiar with binomial distributions). At some points it deals with calculus of variations. The appendix on calculus of variations should be enough though.

Improve this answer
$\endgroup$
1
  • $\begingroup$ That's what I ended up going with after a bunch of recommendations, I was just very hesitant because Bishop's book on machine learning, while heralded by some, is supposed to be a very difficult book to learn from if you don't know it already. $\endgroup$ – Steve P. Jul 15 '13 at 15:02
5
$\begingroup$

The math component would likely include advanced algebra, trig, linear algebra, and calculus at minimum.

But also think outside the box. Good programming skills are also necessary including solid foundations in algorithms (Coursera has two courses on algorithms) and proficiency with MatLab, Octave, or R (and with a flexible programming language like Java, C/C++, or Python). I mention these in response to your question because they are more "applied math" skills in my opinion--and are fundamental to translating between theory and applied implementations.

I have taken a number of the Coursera courses related to machine learning (and agree with one other poster that Prof. Ng's Machine Learning is fantastic) and NN. A few months ago, Coursera hosted a Neural Networks Course (not sure if this is still available) through the University of Toronto and Geoffrey Hinton. A great course and demanded: knowledge of calculus, proficiency with Octave (an open source MatLab-like clone), good algorithmic design (for scalability), and linear algebra.

You might also (while not math per se), think about topics such as natural language processing (for feature extraction, etc.), information retrieval, statistics/probability theory as well as other areas of Machine Learning (to get more theory). Recent texts such as Foundations of Machine Learning (Mohri) or Introduction to Machine Learning (Alpaydin) might be helpful to you in bridging the theory-to-implementation complexity (just in my opinion, this can be a hard leap)--and both texts are very math heavy, especially Foundations.

Again, I think all relate to math and NN but in a broader sense.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks. I majored in applied math as an undergrad (and have extensive programming experience), so I have all of that, except for a rigorous course in abstract algebra, which I'm teaching myself...I ended up going with Neural Networks for Pattern Recognition, by Bishop. For anyone else who's interested, I highly recommend it... $\endgroup$ – Steve P. Jul 22 '13 at 15:27
3
$\begingroup$

See: http://www.quora.com/Career-Advice/How-do-I-become-a-data-scientist Second answer. Pretty complete roadmap.

Gradual intro to Machine learning: follow this excellent Machine Learning 101 course by Andrew Ng of Standford. Did I way it's awesome?

https://www.coursera.org/course/ml

Improve this answer
$\endgroup$
2
$\begingroup$

A very good book (not really introductory, but do not suppose prior knowledge in neural networks) is Brian Ripley: "Pattern Recognition and Neural Networks", which I would say contain much of its prelims. With a BS in applied maths you should be prepared.

Improve this answer
$\endgroup$
0
2
$\begingroup$

MAIN topic is statistics

multivariable calculus

numerical linear algebra (sparse matrices etc) numerical optimisation ( gradient descent etc, quadratic programming)

you might want to read up on gaussian processes and the maths required there try and do some image processing/natural language processing classes

Improve this answer
$\endgroup$
1
  • $\begingroup$ I'm actually taking a NLP course in the Fall. $\endgroup$ – Steve P. Jul 13 '13 at 21:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.