# Why logarithmic scale for hyper-parameter optimization?

I'm using random search for hyper-parameter optimization of a machine learning pipeline. For example, for the C and gamma parameter it is recommended to use logarithmically spaced values. Why should I use such values? For example, if I use logarithmic spaced values from $2^{-5}$ to $2^{15}$, then there will be many more values near to $2^{-5}$ (i.e. near zero) than near to $2^{15}$.

• Why not make draws from an extreme value distribution such as Tweedie, Cauchy, and so on (see en.wikipedia.org/wiki/Tweedie_distribution#Examples)? This would almost certainly provide a better fit to heavy-tailed information and their optimization. – DJohnson Feb 3 '17 at 15:46

• I think the first paragraph is incorrect. Suppose your budget is $k$ different values of $C$. We can search the space $[1,10^3]$ by checking $k$ points equally spaced on a linear scale. If the space is expanded to $[1,10^6]$, this is still true, we can still check $k$ values, except they'll be farther apart. But your point that the difference in performance is more important on a log scale is the heart of it: performance at $C=10\approx11$, but $C=100$ will be different. – Sycorax Feb 3 '17 at 15:54