All Questions

Tagged with
3 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
214 views

Gradient-informed global optimization

I am looking for a review or comparison of global optimization techniques where the gradient of the function is available and utilized to speed up search, like the following: A hybrid descent method ...
1
vote
0answers
204 views

How to combine gradient noise with optimization methods like Adam

I see two options: Apply gradient noise before you apply an optimization method such as Adam (or just SGD with momentum or something else). I.e. you calculate the gradients, then you add noise to it, ...
1
vote
0answers
69 views

Gradient of a sum of indicators

EDITED w.r.t. whuber's comment: Say I have a function $\mathbb R^n \rightarrow \mathbb R$: $$f(w_1,\ldots,w_n) = \frac{n^-\sum_{i\in I^-}w_ix_i}{n^+\sum_{i\in I^+}w_ix_i}$$ with fixed $x_i\in\mathbb ...