Suppose we want to solve the following: $$ \min_{w} f(x^Tw, y) + \lambda g(x^Tw) $$

with $f$ a (logistic) loss and $g$ something like a variance.

Is this a convex optimization problem ?

What are the constraints on $g$ to make it convex ?

PS: the original intent is to constrain the problem so that predicted values have low variance; so any hint on variance instead of sparsity regularization are welcome


If $\lambda \geq 0$, and $g(x^{T}w)=(x^{T}w)^{2}=w^{T}(xx^{T})w$, then $\lambda g(x^{T}w)$ is clearly a convex function of $w$. If you could be more specific about what you want to do with $g()$, then perhaps I could provide a more complete answer.

| cite | improve this answer | |
  • $\begingroup$ the idea would be that $g(x,w) = var(x^Tw)$ $\endgroup$ – oDDsKooL Dec 23 '14 at 13:34
  • $\begingroup$ $x^{T}w$ is a scalar rather than a random variable, so it isn't clear what you mean by its variance? $\endgroup$ – Brian Borchers Dec 23 '14 at 16:58
  • $\begingroup$ sorry my wording was ambiguous; what I seek is a formulation that minimize variance of the predicted values $\hat{y}$. I know how to estimate the bias and variance of a fitted estimator but I'd like to incorporate the low variance constraint in the optimization. $\endgroup$ – oDDsKooL Dec 31 '14 at 7:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.