I have modeling the problem with the following equation: $$ \min_{X} L(X)=f(X)-\alpha g(X) + \beta k(X) $$ where $\alpha \gt 0, \beta \gt 0$, and, in my cases, I found the $f(X)-\alpha g(X) \lt 0$ and $k(X) \gt 0$, it seems this problem is not convex, how to set the parameters $\alpha$ and $\beta$? Is there any optimization algorithm for this problem? Is the Frank-Wolfe algorithm for solving these problems? Thanks!

  • $\begingroup$ What are the forms of $f(X)$, $g(X)$ and $k(X)$? $\endgroup$ – usεr11852 Sep 13 '14 at 13:05
  • $\begingroup$ They are all convex functions. Specifically, $f(X)$ is within-class measure, $g(X)$ is between-class measure, $k(X)$ is a regularization term. $\endgroup$ – mining Sep 13 '14 at 13:18
  • $\begingroup$ The sum of convex functions is convex. Do you worry for multiple local minima (for fixed $\alpha$, $\beta$)? Did a standard optimization procedure (eg. L-BFGS-B) failed? Smoothness maybe? This thread might give you some ideas about $\beta$: Regularized fit from summarized data: choosing the parameter $\endgroup$ – usεr11852 Sep 13 '14 at 13:35
  • 1
    $\begingroup$ Your function is a "difference of convex functions", for which there are specialized branch and bound algorithms. If $X$ is quite small, then it might be reasonable to optimize this objective as a difference of convex functions. If $X$ is large, then this is not likely to be practical. $\endgroup$ – Brian Borchers Sep 13 '14 at 18:23
  • 1
    $\begingroup$ This is discussed for example in the recent book on Global Optimization by Locatelli and Schoen. Your instances are large enough that this approach may not be practical. $\endgroup$ – Brian Borchers Sep 14 '14 at 14:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.