Is it reasonable to study neural networks without mathematical education?

Question

Given the modern state of machine learning technologies and tools (e.g. TensorFlow, Theano, etc.), it seems like entry threshold have recently lowered and it is enough to be able to program on, say, Python, to build interesting things. Another source that supports this point is Machine Learning Specialization on Coursera, that states the following in their FAQ:

What background knowledge is necessary?

You should have some experience with computer programming; most assignments in this Specialization will use the Python programming language. This Specialization is designed specifically for scientists and software developers who want to expand their skills into data science and machine learning, but is appropriate for anyone with basic math and programming skills and an interest in deriving intelligence from data.

On the other hand, there are plenty of other on-line courses (e.g. Stanford Machine Learning on Coursera or Google's Deep Learning on Udacity), as well as classical books, like S. Haykin, Neural Networks: A Comprehensive Foundation, packed with mathematics. Even though I was studying math for several years in university, including statistics, matrices, integral calculus and so on, it's been so long unused that I feel despair by merely looking at those equations. Even Concrete Mathematics by Knuth is discerned at such slow pace that it is seems impossible to finish it at all.

Thus, the following questions arise:

1. Is it reasonable for someone with shallow knowledge in math but programming skills to dive into neural network/machine learning studies?
2. Is it possible to build interesting projects in this area, like those playing atari, using only high-level tools?

Or would it take so much time, that it is better not to make oneself struggle and do something else?

Answer 1

It depends upon your type of Work:

Maths is required if you are working in an applied Science role, i.e. you try to experiments with the known things in Hand i.e. try word embeddings, may be with CNN, and see if the results are good or NOT.

On the other hand , a lot of maths is required if you want to end up as a research scientist, example finding new ways to represent word embedding, or improve the existing word embeddings itself, in case of Text mining.

On the other hand, If you are working as a Software engineer in Machine Learning OR a Machine Learning Engineer, then you just need to train the models using existing knowledge of doing things and Tune it for better performance.

There is a trade off between research and Engineering. More towards Research is More Maths, but More towards Engineering is lesser Maths and More on performance of system in Production.

Another Example to explain would be, for chat Bots. Research Scientist with Maths background require to write a paper for a new models like how LSTM works and can be Used. A applied scientiest will try out a business problem like building a chat bot with LSTM first and publish papers how it worked for them in Labs. A Machine Learning Engineer will replicate the concept which the Applied Scientist had published for their engineering Work (i.e. need to understand maths of paper and replicate it in code, that's it.)

Hope this helps with respect to requirement of knowledge of Maths in Machine Learning

Answer 2

Google is having courses on Deep Learning to train its employees. Given that most of them are in the situation you describe (not much math experience, but good software skills) I would say it's proof that you can profit from Deep Learning without excelling at math.

Now there are tons of tools and sample codes online on lots of cool deep learning projects so it's easy to get started and play with them. For example there's tensorflow which already doesn't require much knowledge of how the backpropagation algorithm works, but there are even simpler layers build on top of it that require even less Deep Learning/theory knowledge, such as Keras.

If you want to build your own you have to keep in mind that you will sometimes need lots of data and lots of computing power for a subset of those projects. (For example, at MIT's Machine Learning class lots of students wanted to replicate the atari project you mentioned, but the TAs suggested not to because of lack of Google-level compute power).

Things you will probably be able to do with 1-2 days of effort include:

• Build your own, relatively simple, architecture.
• Train a complex architecture (using online code) on a dataset of your interest.

Finally, if what you want is to create your new cool architecture that overperforms the state of the art I would say this will be hard to do without understanding the math behind it well.

Answer 3

I'm a PhD candidate in Computational Neuroscience and I work with this kind of software and stuff every day. We also have many students coming in and doing their projects in this field. So I have a bit of experience.

Is it reasonable for someone with shallow knowledge in math but programming skills to dive into neural network/machine learning studies?

Yes, it is. You can use high level abstractions like Keras and get started right away. In my opinion you do not need to know the exact dynamics of the ANN to use it.

As with everything it depends highly on you and how much time and effort you want to invest. I'd say that you need a bit of math to understand the basics of it. An example is the activation functions in a neural net. They play a crucial role but are simple to understand.

If you want to get to the depths of it and really understand how and why it works you'll need pretty extensive math skills. There is no way around that. What I mean are advanced probability theory, advanced calculus and advanced algebra. For example, take a look at the Backpropagation Algorithm.. For more about maths in machine learning you can read this blog post.

Is it possible to build interesting projects in this area, like those playing atari, using only high-level tools?

Yes, it is. There are many great tools out there that allow you to do this. Of course as soon as you'd try to program something new you would hit your limits quite soon, for example, if you wanted to modify a neural network or an algorithm such that it improves or gets faster.

As for Atari I suggest you read this nice blog post. It explains everything in such a detail that you can implement it but still shallow enough to understand it.

So to add to the other answers: I've seen students coming in with low/basic maths skills but good programming skills and they were all able to implement, test and run a solid machine learning pipeline. Here a pipeline means gathering data, preprocessing, training and evaluation of the system as a whole.

So, yes, you can do it.

Answer 4

I'll limit my answer to neural networks.

Is it reasonable for someone with shallow knowledge in math but programming skills to dive into neural network/machine learning studies?

It's reasonable and possible. Here are few reasons to support this conclusion:

1. Neural networks are inspired by the functioning of our brains. Therefore lots of concepts are familiar and easy to understand: neurons, connections, activation etc. This makes the introduction to neural networks smooth and exciting, and doesn't require any math.

2. The basic operation of a neural network, regardless of its size, is easy to understand: forward passing, signals flowing from one level to another, neuron activation etc. Not too much math required here: wighted sums, and non-linear functions such as sigmoid.

3. The math underlying some of the most fundamental algorithms for training neural networks (e.g., back propagation) is not complex: sums, logarithms, multiplications and divisions. And you calculate values that have concrete meaning: error cost, gradient etc. Some other ML techniques require calculation of intermediary values (often matrices) whose meaning is not that clear or intuitive.

4. It's often not the more sophisticated math but rather understanding and experience with the basics that let you dive into more advanced topics such as regularisation, pre-training, dropout etc.

To make things clear there're more complex network architectures and mathematically demanding algorithms for training neural networks. Also, although calculations involved in back propagation are simple its derivation is complex for someone who didn't study calculus. Still the difficulty of derivation or existence of more complex algorithms doesn't mean that studying neural networks is unreasonable. These issues won't prevent you from building good understanding and making practical use of neural networks.

Is it possible to build interesting projects in this area, like those playing atari, using only high-level tools?

In the recent years the interest in neural networks (deep learning especially) is spiking. This resulted in the creation of many good tools and libraries as you observed, some of which are coming from factories such as Google, Microsoft or NVidia. No doubt the quality of these is sufficient to create interesting projects. What might prove more challenging is getting the right amount of quality data to train your network (given such data is not currently available).

Answer 5

It depends. I assume you understand backpropagation algorithm, as it is used by most of NN architectures, and CNNs and RNNs and all that stuff is not that hard if you know backpropagation.

On the one hand, Theano/Tensorflow don't seem to be a good example since they are basically DSLs to write matrix/tensor computations, and they're pretty math-heavy and low-level (you write down actual mathematical operations, and not only use fit, transform/predict api).

On the other hand, there is Keras and scikit-learn, which only have high-level apis, and don't require as much plumbing.

In particular, in Keras you can use some models that were pretrained or have a predefined architecture that is known to work for some problems.

Of course if you don't have good math knowledge you can run into problems with black-box models, but if you just want to apply stuff that seems to work with something to some other thing, then it's definitely possible, for example check out this simple project.

You might also be interested in Creative Applications of Deep Learning with TensorFlow as it seems to be aimed at nontechnical people (I don't know if it's feasible to learn it without any technical background, but at least it contains lots of cool examples).

Answer 6

The essence of a neural network is the graph. While graphs may be part of math, their concepts are as old as relationship itself and pre-date it.

If it were necessary for learning to be complex, then brains probably wouldn't have evolved at all, for the probability of ordered complexity is too improbable.

So, then, one must ask: what is the simplest machine that can learn? But, of course, that begs the epistemological question: what is learning?

Learning is the juxtaposition of two novel states, forming a memory. And there you have the basis of AI: memory.