Preference learning with Bayesian optimization

Question

I want to learn parameter preferences of users for different algorithms. The users are queried for their preference for one of the visualizations generated from a pair of parameter configurations for an algorithm.

So I have some data, that contain a lot of tuples like $(X, Y, D)$, where $X$ and $Y$ are parameter configurations and $D = 1$, if $X$ is preferred to $Y$, otherwise $D = 0$.

From this data, I want to be somehow able to apply Bayesian optimization with Gaussian processes.

Does some implementation for that already exist?

Answer 1

This solution does not use the Bayesian optimization with Gaussian processes, but it could help. I would transform you data from three columns $(X,Y,D)$ just into columns of them $(A,B)$ containing pairs $(X,D)$ and $(Y,1-D)$. $A$ data are from $\mathbb{R}^n$ and $B$ are binary.

Now, you can use a classification algorithm that would return the probabilities $P(B=0|A)$ and $P(B=1|A)$. Having them, you can maximize this $$ \frac{P(B=1|A)}{P(B=0|A)} $$ To find the adopting any optimization approach (gradient-based or evolutionary). This would maximize the probability that your $A^*$ will be better in comparison to any other $A$.