Search Results
| Search type | Search syntax |
|---|---|
| Tags | [tag] |
| Exact | "words here" |
| Author |
user:1234 user:me (yours) |
| Score |
score:3 (3+) score:0 (none) |
| Answers |
answers:3 (3+) answers:0 (none) isaccepted:yes hasaccepted:no inquestion:1234 |
| Views | views:250 |
| Code | code:"if (foo != bar)" |
| Sections |
title:apples body:"apples oranges" |
| URL | url:"*.example.com" |
| Saves | in:saves |
| Status |
closed:yes duplicate:no migrated:no wiki:no |
| Types |
is:question is:answer |
| Exclude |
-[tag] -apples |
| For more details on advanced search visit our help page | |
Results tagged with neural-networks
Search options not deleted
user 105137
Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.
72
votes
What exactly are keys, queries, and values in attention mechanisms?
Big picture
Basically Transformer builds a graph network where a node is a position-encoded token in a sequence.
During training:
Get un-connected tokens as a sequence (e.g. sentence).
Wires connect …
- 1,568
0
votes
What's the difference between variance scaling initializer and xavier initializer?
To understand the difference of each initialization, we need to undersetand what is going on inside the neural network (NN) forward and backward propagation and how to manage the neuron output signals …
- 1,568
0
votes
Why are the embeddings of tokens multiplied by $\sqrt D$ (note not divided by square root of...
In deep neural network layers, we want to retain the signals having normal distributions, and avoid diminishing and exploding gradients. This is the primary goal of weight initialization such as Xavie …
- 1,568
0
votes
Multi-Head attention mechanism in transformer and need of feed forward neural network
The token (query) to token (key) connection in the self attention mechanism captures word to word relation. But how about the relation between the parts of the words?
Jane played piano and John dance …
- 1,568
0
votes
YOLO loss function width and height component explanation
YOLOv1 from Scratch explains as below. It also explains other considerations made in the loss function design.
Lets say we have a very large bounding box and we take those subtracts and squared, that …
- 1,568
0
votes
yolo cost function
if we have negative values for the width and height
As in the original paper, the bounding box size is normalised between 0 and 1, hence will not be negative.
- 1,568
0
votes
Matrix form of backpropagation with batch normalization
In Python as explained in Understanding the backward pass through Batch Normalization Layer.
cs231n 2020 lecture 7 slide pdf
cs231n 2020 assignment 2 Batch Normalization
Forward
def batchnorm_for …
- 1,568