pir
  • Member for 8 years, 4 months
  • Last seen more than a month ago
6 answers
47 votes
45k views
Importance of local response normalization in CNN
Accepted answer
23 votes

It seems that these kinds of layers have a minimal impact and are not used any more. Basically, their role have been outplayed by other regularization techniques (such as dropout and batch ...

View answer
3 answers
14 votes
20k views
Non-linearity before final Softmax layer in a convolutional neural network
Accepted answer
15 votes

You should not use a non-linearity for the last layer before the softmax classification. The ReLU non-linearity (used now almost exclusively) will in this case simply throw away information without ...

View answer
3 answers
6 votes
12k views
Oscillating validation accuracy for a convolutional neural network?
7 votes

This is likely due to the ordering of your dataset. If there's many observations of the same class in a sequence the weights of the network will move too far in the direction of classifying this class....

View answer
1 answers
5 votes
579 views
Can deep neural nets offer state of the art results on regression problems?
4 votes

Deep neural networks such as CNNs or RNN/LSTMs are only truly effective if the data has a structure that can be exploited in the modeling. For instance, this can be spatial (pixels in images) or ...

View answer
2 answers
5 votes
1k views
Ensemble classifier methods: should we use the class probabilties or the classification itself in stacking models?
4 votes

The answer by mttk is quite nice. I will just add that you could also extract the probabilities and use them as input to a meta-classifier such as a simple logistic regression. This will automatically ...

View answer
2 answers
2 votes
11k views
Computational Complexity of Prediction using SVM and NN?
3 votes

For any computer vision task a convolutional neural network (even a simple one) will beat any kind of SVM. This already happened in 2012 when a ConvNet cut the error rate of previous state of the art ...

View answer
1 answers
5 votes
538 views
Is a Restricted Boltzmann Machine appropriate for predicting a vector?
3 votes

Woaw, this is a really interesting problem. Based on what you write below it does not seem like you want a generative algorithm, but instead a discriminative algorithm (see this post). A generative ...

View answer
1 answers
0 votes
303 views
What does the equation $h^k = \sigma(x * W^K + b^k)$ mean in the context of convolutional neural networks (CNNs)?
2 votes

I will be answering in terms of a usual CNN that is applied to an image. $x$ is the input i.e. the pixels. A convolutional layer in a CNN works by passing over a filter over the data. Here each ...

View answer
2 answers
0 votes
4k views
Convolutional neural network with non-image input data
2 votes

Nothing in the CNN method requires clipping of the input data - that is simply a design choice by the modeller. I am not completely sure what you are asking for, but it seems that you are interested ...

View answer
1 answers
1 votes
431 views
Generative Adversarial Network: How to find the most similar image to the output within the training samples?
Accepted answer
1 votes

This is not a trivial task and to my knowledge, there's no clearly superior way of doing this. You can't just use nearest neighbours in pixel space as the GAN can easily just have shifted the output a ...

View answer
2 answers
9 votes
6k views
Best use of LSTM for within sequence event prediction
1 votes

The most important part is how you "phrase" the classification problem, meaning how you represent the input and what you want to output. Seeing as you have so many different event types you need to ...

View answer
1 answers
3 votes
62 views
Norm-bounded input
Accepted answer
1 votes

From Andrew Ng's lecture nodes at web.stanford.edu/class/cs294a/sparseAutoencoder.pdf it is explained that the norm-bounded input is used to get a non-trivial answer. That makes sense as otherwise all ...

View answer
3 answers
16 votes
11k views
How exactly do convolutional neural networks use convolution in place of matrix multiplication?
0 votes

No, that is not how it is supposed to work. The convolution operation always make the input smaller (for filters with size > 1), not larger as in your example. It is simply an elementwise ...

View answer