Questions tagged [non-convex]

The tag has no usage guidance.

8 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
59 views

Optimization textbooks for statistics and data analytics

Any statistical analysis, machine learning or data science involves some sort of optimization at the end of the day. I'm looking for good linear and nonlinear optimization textbooks for self ...
2
votes
0answers
27 views

How to solve a non-convex with equality constraint optimization problem?

I have a non-convex optimization problem with equality constraint, I can derive the KKT conditions, but it seems just one of the KKT conditions is valid. Could you please give some advice on how to ...
1
vote
0answers
86 views

How to optimize ratiometric loss function with variance term in it?

I'm training a neural network (or any ML model with non-convex gradient-based optimization) to predict a continuous outcome variable. Currently, I use the mean squared error loss function, i.e., if $y$...
1
vote
0answers
28 views

Minimal requirement for reaching a feasible solution in non-convex constrained gradient descent

The problem is as follows: $\max_x f(x) \enspace , \enspace \text{s.t.} \enspace g(x) \leq \alpha $ We can not assume that either $f$ nor $g$ are convex, on the contrary - we can assume they are ...
0
votes
0answers
72 views

Bayesian interpretation of gradient clipping

In the context of Bayesian interpretations of SGD for neural network training, is there an interpretation for the gradient clipping operation which is often included?
0
votes
0answers
20 views

Solving constrained optimization problems with Adam

The adam algorithm has been very successful for solving non-convex optimization problems that appear in deep learning. Are there ways to extend adam to solve constrained optimization problems? Among ...
0
votes
0answers
10 views

Stochastic gradient variance reduced methods

I'm doing stochastic gradient descent on a non-convex optimization problem. Gradient corresponds to an intractable expectation which I approximate via Monte Carlo averaging. I'm trying to infer the ...
0
votes
0answers
46 views

How to solve a DC (Difference of Convex) program?

I have an objective function which is the difference of two convex functions in Tensorflow and I want to minimize it. Formally, I have the following problem: $\text{min}_{x \in \mathcal{X}} \;\;f(x)-...