Questions tagged [residual-networks]
A type of neural network architecture suggested in 2015 that allows to train very deep networks with 100s or 1000s of layers.
0
votes
1answer
16 views
Why resolution is not important for pre-trained models
As far as I understand (and even successfully applied in Kaggle competition), it's possible to feed images of any resolution into the pre-trained model (e.g. ResNet34).
But I do not understand, why it ...
0
votes
1answer
23 views
Residual Network: Is small deviation of layer response good? What's the point of resnet?
I have 2 questions...
In the paper Deep Residual Learning for Image Recognition, it says
We show
by experiments (Fig. 7) that the learned residual functions in
general have small responses, ...
0
votes
1answer
88 views
ResNet 34 training with custom dataset
I am a beginner in Neural Networks and wanted to implement ResNet34 for a pet project at my workplace. Due to confidentiality issues, I do not want to use ImageNet trained weights.
I have a dataset ...
0
votes
1answer
81 views
How does backpropagation in residual network works?
i am studying the deep residual network currently, and i cannot fully understand how backpropagation in residual network works. Here some parts of the paper that i read.
so, how does the ...
1
vote
1answer
153 views
ResNet: What is the content of the second skip-connection?
I have a question regarding the second skip connection in ResNet.
Here is a part of the image of the architecture as it was presented in the paper:
As I understand the output of the pool layer gets ...
5
votes
1answer
91 views
Can a residual neural network be interpreted as a form of ensemble learning?
I stumbled over this paper: "Veit, A., Wilber, M. & Belongie, S. Residual Networks Behave Like Ensembles of Relatively Shallow Networks. (2016).". In it, they argue that residual neural ...
2
votes
0answers
25 views
How to initialize weights in the presence of skip connections?
Weight initialization is an important parameter for success in large networks, in the absence of techniques such as batch normalization that reduces their impact.
There are known initialization ...
3
votes
1answer
250 views
How do the residual blocks prevent exploding gradients?
I am reading Roger Grosse's lecture notes on ResNet and I have a question regarding the explanation on how residue blocks prevent gradient explosion, see the screenshot below:
My confusion is: this ...
0
votes
1answer
125 views
Residual connection in Xception [closed]
I am trying to understand residual connection in Xception.
If I am getting right, there's nothing really happen in residual connection (right figure) because it is just addition.
But, I could not ...
1
vote
1answer
1k views
How resNet increasing the dimension?
In the above image, It is the part of the resNet Architecture, here they have used dotted line to increase the dimension, but my question is How they are increasing the dimension?? or this dotted line ...
0
votes
0answers
45 views
Xception in torch: SpatialDepthWiseConvolution backpropagation error
I'm trying to implement NN inspired by Xception ideas. Can't understand what is wrong with my model...
...
3
votes
1answer
5k views
What are “residual connections” in RNNs?
In Google's paper Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation, it is stated
Our LSTM RNNs have $8$ layers, with residual connections between ...
2
votes
1answer
697 views
Residual network: why is each block learning residual error with respect to identity mapping?
In original version of neural network, it learns H(x) from input x, and in the residual network, it is said that learning is improved by learning only residual error with respect to identity mapping, ...
3
votes
2answers
2k views
Shortcut connections in ResNet with different spatial sizes
If I take Fig.3 of the paper "Deep residual learning for image recognition", and look at the following piece of the residual network:
$3\times3$ conv, 64 filters
...
1
vote
0answers
86 views
Validation error smaller than training error in Deep Residual Learning for Image Recognition paper
At one of the examples of Deep Residual Learning for Image Recognition paper, figure 4, leftmost graph it's said that thin curves denote training error and bold curves validation error.
Throughout ...
0
votes
1answer
262 views
How neural networks manages to learn 0 class for sigmoidal Cross-Entropy loss function using ReLU as activation unit?
Let's say we have binary classification problem: 0 vs 1. Some set of images needs to be mapped to those labels by convolutional neural network.
Main trend for now is to use ReLu as activation unit, ...
4
votes
1answer
356 views
Are shortcut connections with stride > 1 still “identity mappings” in ResNets?
In Deep Residual Learning for Image Recognition, I am trying to understand better the "dotted shortcuts" from Figure 3, where the first convolutional layer in those shortcuts is applied with stride of ...
12
votes
1answer
7k views
What exactly is a Residual Learning block in the context of Deep Residual Networks in Deep Learning?
I was reading the paper Deep Residual Learning for Image Recognition and I had difficulties understanding with 100% certainty what a residual block entails computationally. Reading their paper they ...
3
votes
2answers
1k views
Why residual networks works?
I have few questions, that apperad reading through paper:
Building block of residual network can be viewed as following: data passed to right branch -> convolution, scaling, convolution and in right ...
3
votes
2answers
357 views
Do ResNets from Microsoft experts represent convolutional neural nets?
I am talking about the article of Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun (Microsoft Research team):
"Deep Residual Learning for Image Recognitione (2015)"
ResNets won the 1-st place at ...
4
votes
2answers
847 views
How does the Identity connection in ResNets work
I am currently going through the Research paper "Deep Learning for Image recognition" by Kaiming He. I don't quite understand the concept of shortcut connections.
Suppose the input to the residual ...
10
votes
2answers
867 views
Are Residual Networks related to Gradient Boosting?
Recently, we saw the emergence of the Residual Neural Net, wherein,
each layer consists of a computational module $c_i$ and a shortcut connection that preserves the input to the layer such as the ...
3
votes
1answer
606 views
Residual network dimension changing blocks identity function
In trying to implement ResNet with bottleneck blocks for myself, I got very confused about the identity function residual blocks with different dimensions. They compared identity, conv projections on ...
