I have used deep learning to obtain an multi output objective function (or black box). So it's multi input -> multi output.

Now I need to optimize one of the outputs to get max output.

I managed to get some ans using black box optimization libaries like hyperopt, changing the problem slightly from multi input -> single input.

But what if one of my constraints now involves one of the output?

So e.g. input i1,i2,i3 -> output o1,o2,o3

I need to get max o1, but at the same time o2 < 10.

Is there any tool or library which has this capability?

Thanks for reading.


2 Answers 2


I think I found an ans which can be used with hyperopt.

So supposed my inputs are ap1 - ap6, outputs are o1,o2,o3.

My objective is to get max o1, but constraints are:

ap3 - ap6 = 0.18, o3 < o2. I use inequality 0.17 < ap3 - ap6 < 0.19 to relax the problem a little bit

I have managed to use hyperopt to get my ans as follows:

def objective(args):
    ap1,ap2,ap3,ap4,ap5,ap6 = args
    x_in = np.asarray([ap1,ap2,ap3,ap4,ap5,ap6])
    x_in = x_in.reshape(-1, 1).T

    if (ap3 - ap6 < 0.17) | (ap3 - ap6 > 0.19):
        o1= 1e10
        ans = -np.array(model.predict(x_in))
        o2= np.squeeze(aoa_cl_cruise_scaler.inverse_transform(ans[2,0,0].reshape(-1, 1)))
        o3= np.squeeze(aoa_max_c_end_scaler.inverse_transform(ans[3,0,0].reshape(-1, 1)))
        if o2> o3:
            o1= ans[1,0,0]
            o1= 1e10
    return {'loss': o1, 'status': STATUS_OK }

Is this the best way to solve the problem? It seems that I'm wasting a lot of trials because I can't use hp.uniform to limit their values.

My obj function is a black box obtained thru deep learning. Is it better a retrain a another model with o2,o3 becoming one of the inputs? Not sure if that would help.


You could try an evolution strategy algorithm, e.g. CMA-ES. On a high level it would work like this:

  1. generate $N$ candidate inputs from a gaussian distribution using the input vector of your already calculated single max output as mean (or random initialization) and arbitrary variance.
  2. Look which input vectors satisfy your constraint ($o_2 < 10$) and select the $n$ with highest $o_1$.
  3. Calculate a new mean and variance for your sampling distribution from the $n$ vectors. Repeat until happy.

That's some kinda brute-force approach to your problem. But I hope it helps you any further.

  • $\begingroup$ Thanks Tinu! I'll take a look. $\endgroup$
    – quarkz
    May 22, 2020 at 9:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.