I have a minimisation problem in which the parameters are a mix of integers and scalars. Some of the integers have a small range, around 0-10 but others range in the thousands. To give some context, these are all hyper-parameters of machine learning algorithms. At each iteration for a given set of parameters a series of CV and/or bootstrap tests are done to assess the merit of the parameter set.
Currently I am doing this using standard canned minimisation algorithms (specifically Nelder-Mead simplex) and faking the integer variables as continuous by doing two iterations for each value, one with the floor and the other the ceiling of the continuous value and linearly interpolating. It's crude and inefficient but at seems to at least 'work' in that the algorithm converges, eventually.
As you can imagine, the merit function iterations are expensive, so I don't want to be as wasteful as this process requires. I don't know of a standard way to deal with this kind of problem, but I can't imagine that it hasn't been addressed. Are there well known algorithms and implementations for this kind of problem?