How exactly does machine learning theory work/help in practical problems? For past 3 months, I have gone through a machine learning theory course that focuses purely on theory. It covered many obscure concepts, like L1/L2 regularization, ill conditioning，sparsity, VC dimension, gradient descent, logistic regression, entropy，information gain, Euclidean distance—and the math behind it.
The most confusing thing to me is: How exactly is machine learning theory working in practical problems? Is it worth knowing all of the technical details and math proofs?
After reading the theory materials and knowing how the algorithm are built, I still don't know things like "use a less complicated algorithm first", "what kind of algorithm is more complicated", "what kind of algorithm should be chosen to the hypothesis or what kind of problem is learnable", "how to offer a right hyperplane or making good features to fit the model," etc.
 A: The question is too broad to me to answer. 
But theory is always important when come to design the algorithms. Think about how the popular machine learning algorithms such as Neural Network, we need to know what is gradient / how to calculate gradient, to build the model from data. 
If your daily job is downloading programs and run it on your data, without totally understand it, theory may not be very useful. But if you are working on the algorithm design / the person to invent new models, the theory is essential.
In addition, the terms you mentioned (L1/L2 regularization, ill conditioning，sparsity, VC dimension, gradient descent, logistic regression, entropy，information gain) are very important in most piratical problems also.
For example, 


*

*L1 regularization will give a more sparse system, which may save time on production execution. But comparing to L2, it is harder to optimize, this means we need more time to build the model.

*ill conditioning / numerical stability is also very important. Many algorithms will work in theory but not work in practice because computers can only represent a big or small number in certain degree. 
A: This is a good question, I was in this position myself a while back and it feels extremely daunting.
Knowing the theory is very important in using machine learning practically. Without the theory machine learning just becomes black box you can send data into to get an answer. Great if your plotting a set of meaningless x's and y's from an example you don't care about; but terrible if you actually want to make use of the result. A simple example of this would be regression, higher order regression (eg. aX^2+bX+c) will almost always give you a better fit to your data. You can try this with any program for plotting data and finding the line of best fit (excel is what I have in mind). As you increase the order of the polynomial your error, or R squared values approaches 1 (1 being perfect fit). Seems like a no brainer, whak the order up to 100 and get your perfect line of best fit. However if you understand the way regression works you know that as the number of features in the regression (X^2,X^1,X^0) increases you need more data to fit these features. This is especially the case when you're considering extrapolation over interpolation. Understanding the way the model works allows you to use the right type of model for your data, and get useful answers; answers that you understand the scope and limitations of.
So we know why theory matters. But outside of the simple example about how do you know which model to use. It's a tough question.
A good rule of thumb I read many years ago on this site is you want at least 15 times as many data points as you have features, this is to deal with overfitting - but this is not always possible in my experience.
Another good rule is pick a model that feels like the problem. Got a problem with periodic data (regular ups and downs) maybe think about what models you know that can use a sin wave.
A hard rule is don't confuse regression and classification problems, it makes sense to round 1.5 up to 2, it doesn't make sense to round Red up to Blue.
A model with fewer features is simpler. Simpler models tend to optimize more quickly.
This is the most important rule: if you don't really understand the way the model works write down the kind of result you expect (y increases wrt x^2, houses with more rooms sell for more money, etc.) then if your model disagrees either find a really, really good reason why your intuition was wrong or accept that you don't know what has happened and this model isn't for you until you've done more reading.
