# Questions tagged [non-convex]

The tag has no usage guidance.

23 questions
Filter by
Sorted by
Tagged with
86 views

### Why do we need $\gamma>2$ in SCAD penalty?

The SCAD penalty $p(x | \lambda, \gamma)$ from https://myweb.uiowa.edu/pbreheny/7600/s16/notes/2-29.pdf or the paper "Variable Selection via Nonconcave Penalized Likelihood and its Oracle ...
• 203
308 views

• 3,957
253 views

### Does coordinate wise convex function can be optimized more effectively?

I'm currently working on a non-convex function. It's basically a maximum likelihood problem so I'm trying to optimize this function. I know that non-convex optimization frequently reaches local optima ...
409 views

### Convergence under large set of learning rates

What is the interpretation of a stochastic optimization problem where a gradient descent algorithm is converging under a wide range of learning rate schedules (including ones with quite large initial ...
• 954
1 vote
187 views

### How to minimize the sum of Frobenius norm and Nuclear norm

I have to minimize an objective function of the the form : $||X_{s} - Y_{s}D_{s}||_{F}^{2} + ||D_{s}||_{F}^{2} + ||D_{s}||_{*}^{2}$ where $||.||_{F}$ denotes the Frobenius norm and $||.||_{*}$ ...
• 1,956
751 views

### Why are K-means and GMM (Gaussian Mixture Models) not suitable for discovering clusters with non-convexs shapes?

I have seen that mainly here and from a lot of resources that K-means and Hello all! Gaussian mixtures are not suitable for detecting clusters with non-convex shapes. I know that because both ...
• 125
53 views

### If $\ell_0$ regularization can be done via the proximal operator, why are people still using LASSO?

I have just learned that a general framework in constrained optimization is called "proximal gradient optimization". It is interesting that the $\ell_0$ "norm" is also associated with a proximal ...
2k views

### Solving constrained optimization problems with Adam

The adam algorithm has been very successful for solving non-convex optimization problems that appear in deep learning. Are there ways to extend adam to solve constrained optimization problems? Among ...
• 275
57 views

### How to solve a non-convex with equality constraint optimization problem?

I have a non-convex optimization problem with equality constraint, I can derive the KKT conditions, but it seems just one of the KKT conditions is valid. Could you please give some advice on how to ...
• 21
688 views

### Is error function always assumed and convex?

While updating weights of the neural network, most of the algorithms use convex optimisation because of the reason that error is a convex function. My doubt is that whether the convex-ness of error ...
• 205
1 vote
428 views

### How to optimize ratiometric loss function with variance term in it?

I'm training a neural network (or any ML model with non-convex gradient-based optimization) to predict a continuous outcome variable. Currently, I use the mean squared error loss function, i.e., if $y$...
• 11
366 views

### Optimization textbooks for statistics and data analytics

Any statistical analysis, machine learning or data science involves some sort of optimization at the end of the day. I'm looking for good linear and nonlinear optimization textbooks for self ...
1 vote
43 views

### Minimal requirement for reaching a feasible solution in non-convex constrained gradient descent

The problem is as follows: $\max_x f(x) \enspace , \enspace \text{s.t.} \enspace g(x) \leq \alpha$ We can not assume that either $f$ nor $g$ are convex, on the contrary - we can assume they are ...
• 121
181 views

### How to solve a DC (Difference of Convex) program?

I have an objective function which is the difference of two convex functions in Tensorflow and I want to minimize it. Formally, I have the following problem: \$\text{min}_{x \in \mathcal{X}} \;\;f(x)-...
• 71
8k views

### When will gradient descent converge to a critical point or to a local/global minima) for non-convex functions?

What situations do we know of where gradient descent can be shown to converge (either to a critical point or to a local/global minima) for non-convex functions? For SGD on non-convex functions, one ...
• 361