# Optimize black box multi output function with one of the output as constraint

I have used deep learning to obtain an multi output objective function (or black box). So it's multi input -> multi output.

Now I need to optimize one of the outputs to get max output.

I managed to get some ans using black box optimization libaries like hyperopt, changing the problem slightly from multi input -> single input.

But what if one of my constraints now involves one of the output?

So e.g. input i1,i2,i3 -> output o1,o2,o3

I need to get max o1, but at the same time o2 < 10.

Is there any tool or library which has this capability?

I think I found an ans which can be used with hyperopt.

So supposed my inputs are ap1 - ap6, outputs are o1,o2,o3.

My objective is to get max o1, but constraints are:

ap3 - ap6 = 0.18, o3 < o2. I use inequality 0.17 < ap3 - ap6 < 0.19 to relax the problem a little bit

I have managed to use hyperopt to get my ans as follows:

def objective(args):
ap1,ap2,ap3,ap4,ap5,ap6 = args
x_in = np.asarray([ap1,ap2,ap3,ap4,ap5,ap6])
x_in = x_in.reshape(-1, 1).T

if (ap3 - ap6 < 0.17) | (ap3 - ap6 > 0.19):
o1= 1e10
else:
ans = -np.array(model.predict(x_in))
o2= np.squeeze(aoa_cl_cruise_scaler.inverse_transform(ans[2,0,0].reshape(-1, 1)))
o3= np.squeeze(aoa_max_c_end_scaler.inverse_transform(ans[3,0,0].reshape(-1, 1)))
if o2> o3:
o1= ans[1,0,0]
else:
o1= 1e10
return {'loss': o1, 'status': STATUS_OK }


Is this the best way to solve the problem? It seems that I'm wasting a lot of trials because I can't use hp.uniform to limit their values.

My obj function is a black box obtained thru deep learning. Is it better a retrain a another model with o2,o3 becoming one of the inputs? Not sure if that would help.

You could try an evolution strategy algorithm, e.g. CMA-ES. On a high level it would work like this:

1. generate $$N$$ candidate inputs from a gaussian distribution using the input vector of your already calculated single max output as mean (or random initialization) and arbitrary variance.
2. Look which input vectors satisfy your constraint ($$o_2 < 10$$) and select the $$n$$ with highest $$o_1$$.
3. Calculate a new mean and variance for your sampling distribution from the $$n$$ vectors. Repeat until happy.

That's some kinda brute-force approach to your problem. But I hope it helps you any further.

• Thanks Tinu! I'll take a look. May 22, 2020 at 9:09