I am working on a statistical learning exercise that requires some knowledge of convex optimization which I am unfortunately lacking.

Consider the linear regression model $$y_i=h(x_i)^\top\beta+\epsilon_i \quad i=1,\ldots,n$$ where $\varepsilon_i$ are random errors. Consider a symmetric, positive definite kernel $K(x_i, x_j)=h(x_i)^\top h(x_j)$. The estimator $\hat{\beta}$ is the solution to $$\min _\beta\left[\|\beta\|^2+\sum_{i=1}^n \xi_i^2\right]$$ subject to the constraint $\xi_i=y_i-h\left(x_i\right)^{\top} \beta$.

I am now asked to give (i) the Lagrange primal function, (ii) the Wolfe dual function and (iii) to derive an expression for $\hat{\beta}$.

For (i), by plugging in the equality constraint and introducing Lagrangian multipliers I get $L_P=\beta^\top\beta+\sum_{i=1}^n \alpha_i(y_i-h(x_i)^\top\beta)^2$.

For (ii), I compute $$\frac{\partial L_P}{\partial \beta}=\beta+2\sum_{i=1}^n\alpha_iy_ih(x_i)-2\sum_{i=1}^n\alpha_ih(x_i)^\top\beta h(x_i)=0.$$

Now I reckon I must isolate $\beta$ in the above and plug the expression I obtain into the primal problem to obtain the dual. I am stuck here. If my derivation so far is correct, I’d appreciate a hint on how to do this. (If it is not I’d of course be glad to be made aware of any mistake.) Thanks for considering my question!