Optimize black box multi output function with one of the output as constraint I have used deep learning to obtain an multi output objective function (or black box). So it's multi input -> multi output.
Now I need to optimize one of the outputs to get max output.
I managed to get some ans using black box optimization libaries like hyperopt, changing the problem slightly from multi input -> single input.
But what if one of my constraints now involves one of the output?
So e.g. input i1,i2,i3 -> output o1,o2,o3
I need to get max o1, but at the same time o2 < 10.
Is there any tool or library which has this capability?
Thanks for reading.
 A: I think I found an ans which can be used with hyperopt.
So supposed my inputs are ap1 - ap6, outputs are o1,o2,o3.
My objective is to get max o1, but constraints are:
ap3 - ap6 = 0.18, o3 < o2.
I use inequality 0.17 < ap3 - ap6 < 0.19 to relax the problem a little bit
I have managed to use hyperopt to get my ans as follows:
def objective(args):
    ap1,ap2,ap3,ap4,ap5,ap6 = args
    x_in = np.asarray([ap1,ap2,ap3,ap4,ap5,ap6])
    x_in = x_in.reshape(-1, 1).T

    if (ap3 - ap6 < 0.17) | (ap3 - ap6 > 0.19):
        o1= 1e10
    else:
        ans = -np.array(model.predict(x_in))
        o2= np.squeeze(aoa_cl_cruise_scaler.inverse_transform(ans[2,0,0].reshape(-1, 1)))
        o3= np.squeeze(aoa_max_c_end_scaler.inverse_transform(ans[3,0,0].reshape(-1, 1)))
        if o2> o3:
            o1= ans[1,0,0]
        else:
            o1= 1e10
    return {'loss': o1, 'status': STATUS_OK }

Is this the best way to solve the problem? It seems that I'm wasting a lot of trials because I can't use hp.uniform to limit their values.
My obj function is a black box obtained thru deep learning. Is it better a retrain a another model with o2,o3 becoming one of the inputs? Not sure if that would help.
A: You could try an evolution strategy algorithm, e.g. CMA-ES. On a high level it would work like this:


*

*generate $N$ candidate inputs from a gaussian distribution using the input vector of your already calculated single max output as mean (or random initialization) and arbitrary variance.

*Look which input vectors satisfy your constraint ($o_2 < 10$) and select the $n$ with highest $o_1$.

*Calculate a new mean and variance for your sampling distribution from the $n$ vectors. Repeat until happy.


That's some kinda brute-force approach to your problem. But I hope it helps you any further.
