I'm having some trouble getting the NM Simplex to find a good minimum for selecting hyperparameters of a rbf SVC. Not only am I tuning the 2 SVC parameters (C and gamma) I also have five class weights that I'm trying to tune. Grid searches seem fairly infeasible in this case. The NM Simplex doesn't seem test inputs far away enough from the initial guess. I am transforming the inputs (e.g. the NM Simplex is searching in range i while 2^i is inputted as the C parameter for the SVC). Even so, the NM Simplex always seems to end up back at the initial guess.
Any suggestions or any other minimization algorithms that might be worthwhile?