Search Results
| Search type | Search syntax |
|---|---|
| Tags | [tag] |
| Exact | "words here" |
| Author |
user:1234 user:me (yours) |
| Score |
score:3 (3+) score:0 (none) |
| Answers |
answers:3 (3+) answers:0 (none) isaccepted:yes hasaccepted:no inquestion:1234 |
| Views | views:250 |
| Code | code:"if (foo != bar)" |
| Sections |
title:apples body:"apples oranges" |
| URL | url:"*.example.com" |
| Saves | in:saves |
| Status |
closed:yes duplicate:no migrated:no wiki:no |
| Types |
is:question is:answer |
| Exclude |
-[tag] -apples |
| For more details on advanced search visit our help page | |
Results tagged with gradient-descent
Search options not deleted
user 128472
Gradient descent is a first-order iterative optimization algorithm. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or of the approximate gradient) of the function at the current point. For stochastic gradient descent there is also the [sgd] tag.
2
votes
How does ResNet or CNN with skip connections solve the gradient exploding problem?
I'm not 100% sure, but I would guess that this is more referring to normalization like BatchNorm rather than skip connections. It's not like ResNets will not explode without any normalization and not …
- 391
20
votes
How does rectilinear activation function solve the vanishing gradient problem in neural netw...
This is why it's probably a better idea to use PReLU, ELU, or other leaky ReLU-like activations which don't just die off to 0, but which fall to something like 0.1*x when x gets negative to keep learn …
- 391