# Tuning the regression hyperparameters

I am trying to find the hyperparameters of a gaussian process regression algorithm using sklearn. The book (Rasmussen), says I should to maximize the log marginal likelihood given by $$\log(\mathbf{y}|X,\mathbf{\theta})=-\frac{1}{2} \mathbf{y}^TK_y^{-1}\mathbf{y}-\frac{1}{2}\log(\det(K))-\frac{n}{2}\log(2\pi)$$ So I start from a RBF kernel in sklearn with some parameters (can they be easy and random, say just both 1.0?) and then try to find the correct $$\theta$$? I don't understand this approach, should I do this for each label in my dataset in bulk? Or consider one point of my training set at a time and update the weights at each iteration? I apologise for the confused question, but can somebody explain how to start implementing this method?

• Thanks for your reply. My misunderstanding is perhaps at an even more basic level: that optimization formula depends on a single label, so how is the hyperparameter tuning to happen in practice? I implement that formula for a single $y$ and then? How would I even do it for all my training labels "at once"? Oct 18, 2021 at 18:26
• I am completely lost. I supposedly need to find the $\theta$ that minimizes that expression. Are you saying there is something else I should be minimizing? Oct 18, 2021 at 18:34
• Ok, thank you. That formula depends on a single label $y$, so how do I implement this maximization in practice? Do it wrt one $y$, obtain new parameters for the kernel, and do it again on the next $y$ with the new kernel? How would I do it "in bulk"? Oct 18, 2021 at 19:02