# Questions tagged [non-convex]

The tag has no usage guidance.

13 questions
Filter by
Sorted by
Tagged with
2k views

### Gradient descent on non-convex functions

What situations do we know of where gradient descent can be shown to converge (either to a critical point or to a local/global minima) for non-convex functions? For SGD on non-convex functions, one ...
59 views

### Optimization textbooks for statistics and data analytics

Any statistical analysis, machine learning or data science involves some sort of optimization at the end of the day. I'm looking for good linear and nonlinear optimization textbooks for self ...
499 views

### Non-convex optimization without using gradient descent

If we want to optimize a convex function, we could use methods like gradient descent or computing the derivative of the function and equalize to zero so we can obtain the global minimum. But I ...
27 views

### If $\ell_0$ regularization can be done via the proximal operator, why are people still using LASSO?

I have just learned that a general framework in constrained optimization is called "proximal gradient optimization". It is interesting that the $\ell_0$ "norm" is also associated with a proximal ...
27 views

### How to solve a non-convex with equality constraint optimization problem?

I have a non-convex optimization problem with equality constraint, I can derive the KKT conditions, but it seems just one of the KKT conditions is valid. Could you please give some advice on how to ...
89 views

### How to optimize ratiometric loss function with variance term in it?

I'm training a neural network (or any ML model with non-convex gradient-based optimization) to predict a continuous outcome variable. Currently, I use the mean squared error loss function, i.e., if $y$...
23 views

### How to diagnose why numeric solver is not converging? [closed]

Are there specific approaches, methods or software that help in determening why an optimum is not found for a particular optimization problem? For example, the solution landscape could be visualized ...
28 views

### Minimal requirement for reaching a feasible solution in non-convex constrained gradient descent

The problem is as follows: $\max_x f(x) \enspace , \enspace \text{s.t.} \enspace g(x) \leq \alpha$ We can not assume that either $f$ nor $g$ are convex, on the contrary - we can assume they are ...
116 views

### Is error function always assumed and convex?

While updating weights of the neural network, most of the algorithms use convex optimisation because of the reason that error is a convex function. My doubt is that whether the convex-ness of error ...
73 views

### Bayesian interpretation of gradient clipping

In the context of Bayesian interpretations of SGD for neural network training, is there an interpretation for the gradient clipping operation which is often included?
20 views

### Solving constrained optimization problems with Adam

The adam algorithm has been very successful for solving non-convex optimization problems that appear in deep learning. Are there ways to extend adam to solve constrained optimization problems? Among ...