Search Results
| Search type | Search syntax |
|---|---|
| Tags | [tag] |
| Exact | "words here" |
| Author |
user:1234 user:me (yours) |
| Score |
score:3 (3+) score:0 (none) |
| Answers |
answers:3 (3+) answers:0 (none) isaccepted:yes hasaccepted:no inquestion:1234 |
| Views | views:250 |
| Code | code:"if (foo != bar)" |
| Sections |
title:apples body:"apples oranges" |
| URL | url:"*.example.com" |
| Saves | in:saves |
| Status |
closed:yes duplicate:no migrated:no wiki:no |
| Types |
is:question is:answer |
| Exclude |
-[tag] -apples |
| For more details on advanced search visit our help page | |
Results tagged with neural-networks
Search options not deleted
user 215833
Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.
0
votes
multi layer perceptron tested in raw data vs edited data
What architecture is your neural network? Classifying on raw pixels is hard, even for greyscale images of that size. If you're not using a Convolutional neural network or other architecture designed f …
- 1,184
0
votes
Have I designed my network correctly? If not, how do I fix it?
If all of your input graphs are the same size, then your approach works fine. Otherwise, the size of the linear layer would need to change based on the size of the input graph. Pooling is one way of g …
- 1,184
1
vote
Accepted
What happens at the input node in an inception module during the backwards pass?
Yes, you sum. One way to convince yourself is to remember that the behavior of calculating the backward pass is independent at each node, because of the chain rule, so it doesn't even matter that ther …
- 1,184
2
votes
Accepted
Residual block in temporal convolutional neural network
Yes they're the same. The 1x1 convolution is in both of them. For residual mappings, you're adding the old layer's input value to the input of the later layer down the line (aka up the image). If the …
- 1,184
-1
votes
How to train a neural network *not* to give a certain output?
To answer the question about your approach of providing target distributions: yes, that is a correct approach, and softmax is the right final layer for this approach.
- 1,184
2
votes
Accepted
Variable length memory / information flow in Transformers
RNN's operate on the input sequence one at a time going down the line.
Transformers have input width greater than the length of the longest input sequence. It eats up the whole sequence at once, chews …
- 1,184
1
vote
Layer normalization for neural networks
N is the batch size. For layer normalization, normalizing across the rows of the input data means that for each data point in the batch (of which there are N), we normalize the vector of values (which …
- 1,184
43
votes
Accepted
Why Not Prune Your Neural Network?
Pruning is indeed remarkably effective and I think it is pretty commonly used on networks which are "deployed" for use after training.
The catch about pruning is that you can only increase efficiency, …
- 1,184
1
vote
Overlap-tile strategy in U-Nets
Yeah their description is a bit confusing. I agree with your interpretation.
This paper has 15k citations, so if that strategy is effective, it's probably pretty commonly used. Otherwise, my only gues …
- 1,184
1
vote
Accepted
Intuition behind replacing backpropagation with random matrices
This diagram might be useful: (from this paper)
Direct Feedback Alignment (DFA) is a modification of Feedback Alignment (FA), which replaces Backpropogation (BP). With FA, you replace $\mathbf{W_i}$ …
- 1,184
38
votes
What does 1x1 convolution mean in a neural network?
The main reason I didn't understand 1x1 convolutions is because I didn't understand how $any$ convolutions really worked—the key factor is how computing a convolution of multiple channels/filters work …
- 1,184
1
vote
Neural ODEs gradient calculation for multiple time steps
From the paper:
Most ODE solvers have the option to output the state $\mathbf{z}(t)$ at multiple times. When the loss depends
on these intermediate states, the reverse-mode derivative must be broken i …
- 1,184
1
vote
Accepted
LSTM performs poorly with monotonically increasing test set values never seen in training. Why?
You may be suffering from a common issue of neural networks failing to generalize to numerical inputs unseen in training.
The best display of this behavior I know is the figure from this paper:
Capti …
- 1,184
5
votes
Accepted
Intuition behind the use of multiple attention heads
I'll tell you what I know/have heard about it. There's probably other analysis out there I haven't seen.
I think multi-head attention was introduced in the transformer network paper. Their justificati …
- 1,184
1
vote
Why no orthogonality of residuals and predictions in neural networks?
It appears this is due to the instability of the hidden features and the dynamics of gradient descent with these shifting features.
I did some experiments fitting with a small network with 3 hidden ne …
- 1,184