# Preference learning with Bayesian optimization

I want to learn parameter preferences of users for different algorithms. The users are queried for their preference for one of the visualizations generated from a pair of parameter configurations for an algorithm.

So I have some data, that contain a lot of tuples like $(X, Y, D)$, where $X$ and $Y$ are parameter configurations and $D = 1$, if $X$ is preferred to $Y$, otherwise $D = 0$.

From this data, I want to be somehow able to apply Bayesian optimization with Gaussian processes.

Does some implementation for that already exist?

• By implementation you would mean code for a statistics package (which would be off-topic here) or a statistical procedure for your problem at hand?
– Andy
Jul 30, 2015 at 12:09
• Both would be welcome. A statistical procedure would be great, if some code already exists for it, it would be even better. Jul 30, 2015 at 12:23
• Are your data discrete or continuous? More specifically, are X and Y real numbers or vectors or not? Jul 30, 2015 at 13:10
• X and Y are in R^n. So yes, they are vectors. Jul 30, 2015 at 13:16
• You speak about some optimization. What do you want to have optimal? The parameters of your visualization? Jul 30, 2015 at 13:18

This solution does not use the Bayesian optimization with Gaussian processes, but it could help. I would transform you data from three columns $(X,Y,D)$ just into columns of them $(A,B)$ containing pairs $(X,D)$ and $(Y,1-D)$. $A$ data are from $\mathbb{R}^n$ and $B$ are binary.
Now, you can use a classification algorithm that would return the probabilities $P(B=0|A)$ and $P(B=1|A)$. Having them, you can maximize this $$\frac{P(B=1|A)}{P(B=0|A)}$$ To find the adopting any optimization approach (gradient-based or evolutionary). This would maximize the probability that your $A^*$ will be better in comparison to any other $A$.