# What are good alternatives to grid search?

What are good alternatives to grid search? By this I mean some techniques/packages for Automated Machine Learning that can optimize model parameters in 'auto' mode.

First question is what techniques can be applied to solved this task: for example genetic algorithm, bayesian optimization, etc?

Second question is how well it work in practice and what packages can be used for example auto-sklearn, etc?

What they do is to model the probability distribution of the hyperparameter space which reflects on the idea that models yielding good results are concentrated around the true value. Now, for any probability distribution with finite maximum the following holds. Take the region around the maximum which accounts for 5% of the total probability. If you take n samples at random. Each one of them has a 0.05 chance of falling within that interval. With probability $(1-0.05)^n$ they all will miss that area.
Now they ask: how many samples n do I need to take so that with high probability (say 95%), at least one of them falls in that region: $$1-(1-0.05)^n \ge 0.95$$ which yields $n \ge 60$. This algorithm has been used with success in deep learning, where the number of parameters if huge.