ML & AI - Algorithm recommendation for "optimization by tuning parameters"? As a newbie in Machine Learning (ML) and Artificial Intelligence (AI), I am here to ask some general algorithm pointers on my following question, which I have simplified into a very generic question.
I have a model $M$ defined by a few parameters $p_1, p_2, \dots,p_n$. Now I wish to optimize $M$, i.e., maximize one property value of $M$, by tuning $p_1, p_2, \dots,p_n$. I am looking for algorithms that sort of "learn" what is good and what is bad along the way and find the optimal solution in the end.
One similar algorithm in my head is genetic algorithm in AI, but for some reason I cannot use it. So I am here to get some algorithm pointers in ML, hopefully.
 A: Your question is tricky to answer because it covers almost all of machine learning!
In some cases, it's possible to solve for the optimal values directly, using a known formula. Suppose you want to model some data with a line:
$$Y=AX$$
but don't know the slope . The least-squares algorithm can calculate the slopes $A$ directly, without actually searching for them.
In other cases, algorithms have parameters or hyper-parameters that need to be determined empirically. For example, Support Vector Machines have two parameters: $c$ and $\gamma$, and the appropriate values for each depend on problem. There are many different ways to perform hyperparameter optimization. The simplest is grid search: pick a (regularly-spaced) grid of possible values for each parameter, then test them all. 
It is often possible to do lot better than a grid search, however, and some of these approaches do use genetic algorithms or related concepts (e.g., gradient descent or simulated annealing). Some algorithms try to quickly converge on the "best" region of the parameter space, and then spend their time finding the precise location of that optima. 
Others, like Bayesian Global Optimization, try to produce a reasonable "map" of the entire parameter space. 
There are whole books, conferences, and journals on the topic of optimization. Take a look around and feel free to come back with more specific question. 
A: Thats what most machine learning algorithms try to do! You essentially asking us to suggest machine learning algorithms for performing machine learning! 
If you specifically want to understand how optimization in general is done, there are some free convex optimization courses online. Convex optimization is basically the "ideal" case for a problem (easiest to solve), you can branch out to other cases from there. 
Why we might want to optimize a particular model with a particular set of parameters is a whole other very long discussions. The best I can recommend is to just keep taking courses and reading on machine learning & statistics. 
