Bayesian optimization with constraints

Question

I want to perform Bayesian optimization for a certain physical task but with additional requirements. We have access to a set of variables and want to maximize (multiple) signal outputs from an instrument. Our inputs to be optimized can be assumed to be in tidy format. However, for certain features (generally in pairs of triplets) when one column is non-zero we want exactly one other to be non-zero. Another requirement is that in addition to the above we want a subset of x's to sum to unit. For example

$$ \mathcal{D} = (\mathbf{x_1},\mathbf{x_2},\mathbf{x_3},\mathbf{x_4},\mathbf{x_5} ) $$

When any of $\mathbf{x_1-x_4}$ is non-zero exactly one other of the same or at most one can be non-zero be non-zero ($\mathbf{x_5}$ does not have this requirement). For example: $$ \begin{array}{111111} 10 & 90 & 0 & 0 & 20 & \text{Valid} \\ \end{array} $$ $$ \begin{array}{111111} 90 & 10 & 10 & 20 & 0 & \text{Invalid} \\ \end{array} $$ $$ \begin{array}{111111} 00 & 00 & 0 & 0 & 1 & \text{Invalid} \\ \end{array} $$ $$ \begin{array}{111111} 10 & 10 & 10 & 0 & 0 & \text{Invalid} \\ \end{array} $$ The x's in general can be real or even categorical. How can these restrictions be imposed in the context of Bayesian optimization?. From what I can gather, current implementations of bayes-opt could propose any set of these i.e. (10,100,20,90)

Answer 1

Some encoding might solve your problem:

• For the constaint $x_i>0$ iff $x_j>0$, you can introduce an additional variable $y_{i,j}$ that will state if they two are zero or positive.
• For sum to one, you can apply for example normalization after sampling.

So, you will need to wrap your optimized function into something that will decode these codes.

1. Sample all variables (including the additional ones such as $y_{i,j}$.
2. Decode them so the mutual non-negativity and sum to one is ensured.
3. Evaluate the value.