Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

An area of machine learning concerned with learning hierarchical representations of the data, mainly done with deep neural networks.

-1
votes
0answers
11 views

what are the ways to gain good accuracy in deep learning competition apart from changing hidden layer especially in case of image data set? [on hold]

I am newbie in field of deep learning but i have experience in machine learning.like in machine learning competition we do plotting apply some statistics to gain understanding of data and by that we ...
0
votes
0answers
5 views

Vanishing gradient in End-to-End Memory network

I'm implementing a language model using End-To-End Memory Networks, the training starts off fine, but after a while the loss stagnates. I've looked at the gradients and they seem to be too small. What ...
1
vote
1answer
10 views

Reinforcement Learning: definition of expected discounted return in Sutton and Barto's book

I am going through Sutton and Barto's book on reinforcement learning http://incompleteideas.net/book/bookdraft2017nov5.pdf In the book pg 44 equation 3.8, the authors define expected discounted ...
1
vote
0answers
8 views

Higher Order of Vectorization in Backpropagation in Neural Network

I am learning a machine learning class online from Stanford, namely CS 229. There is one section about deep learning and back-propagation in deep learning. The network looks like: The forward ...
0
votes
0answers
8 views

Which algorithms for mixed type datasets (binary classification)?

I am new to machine learning and I am trying to implement a model for a binary classification problem (output class 0 or class 1), and wondering which algorithms I should consider, since my dataset is ...
0
votes
0answers
6 views

resnet 50 - keras - learning rate stopped? [duplicate]

hth do I increase the speed of this? I mean the loss is moving down by hairs. HAIRS. ...
0
votes
0answers
9 views

Maximizing cross entropy reinforcement learning

I have read that in reinforcement learning, maximizing the entropy enables the policy to behave more randomly. My question comes in three parts: (1) In the equation below in the cross-entropy term ...
0
votes
0answers
6 views

Active learning for object detection- Batch Selection

I have a small dataset of about 220 images for three classes. I am using YOLO (you only look once) network for an object detection. I am trying to use Active learning in order to reduce the number of ...
0
votes
0answers
8 views

Best Generative Model for text train data augmentation

I have text training data and want to train an deep NLP network that is doing a discrimination task. For data augmentation I want to use a Generative Model to generate more text data for each label. ...
1
vote
0answers
18 views

BatchNorm after ReLU

I am currently experimenting with different settings for a U-Net (https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/) based image segmentation and I was unable to find out if it makes any ...
0
votes
0answers
7 views

Layer size extension of a pre-trained neural net

I am searching for a good way to deal with a training dataset for a deep net, where some of the input features are not available for some of the input vectors. My goal is to utilize the whole input ...
1
vote
0answers
30 views

Intuitive definition of manifold regularization for neural networks [on hold]

I am studying the deep neural networks and I have been assigned a project on the manifold regularized neural networks, in particular the definition is as follows: "enabling semi-supervised learning ...
2
votes
0answers
15 views

Why is it difficult to learn a single kernel that performs well at all positions in the convolutional feature map?

I am reading Deep Learning book by Ian Goodfellow, in which they wrote (in chapter 9, section 9.5) that: " ... MATLAB refers to this as full convolution, in which enough zeros are added for every ...
1
vote
0answers
24 views

Comparing two models of recurrent neural networks for predicting time series

Disclosure: I have posted this question on Data Science SE, but got no answers and traffic seems to be higher here. If this is inappropriate let me know and I'll remove the question there. I'd like ...
1
vote
0answers
15 views

Is there any Good comprehensive reference for tensor calculus? [duplicate]

I am currently reading this RNN blog, where it talks about Backprop through time. I am struggling to derive it and don't understand how to go about such derivations in general. Stuff like this ends ...
0
votes
1answer
46 views

How to move reinforcement learning model into production?

I have trained reinforcement learning agent on a custom environment using the DQN technique. The custom environment is a simulation of a real production environment. Now I have trained NN model with ...
0
votes
0answers
6 views

MFCC features for word boundary detection

How can the MFCC features extracted from a speech signal be used to perform word/sentence boundary detection?
1
vote
1answer
33 views

Why does my LSTM take so much time to train?

I am trying to train a bidirectional LSTM to do a sequential text-tagging task (particularly, I want to do automatic punctuation). I use letters as the building-blocks: I represent each input letter ...
0
votes
0answers
7 views
1
vote
1answer
33 views

Attention based models for machine comprehension

Currently, I am understanding Attention models. I specifically need it to build a machine comprehension model (a model which can find answer to a question from a given comprehension). But I want to ...
1
vote
1answer
37 views

Bad performance with ReLU activation function on MNIST data set

I'm quite new to neural networks and currently I'm trying to train a non convolutional neural network on the MNIST data set. I'm observing some behaviour I don't quite understand. This is the code ...
0
votes
1answer
31 views

What are the other optimization problems in deep learning other than training neural nets?

I understand training deep neural nets is an optimization problem, however, I do not understand what other problems can be in deep learning that involves optimization? In Deep Learning book by Ian ...
0
votes
0answers
35 views

How to construct two disjoint sets such that $P(x \in S_1) + P(x \in S_2) > 1$ but $S_1 \cap S_2 = \emptyset$ [duplicate]

As Deep Learning mentioned in section 3.12: ... we saw that the probability of a continuous vector-valued $x$ lying in some set $S$ is given by the integral of $P(x)$ over the set $S$. Some choices ...
0
votes
0answers
30 views

2-output node Neural network. Only the first output node can predict accurate enough results

I have a three hidden layer neural network. Input layer has 116 nodes(means I have 116 features in every training data set) and output layer has 2 nodes(means I have 2 labels in every training data ...
0
votes
1answer
49 views

What is a good machine learning academic research workflow/tools? [closed]

I'm interested in ML research and want to get some insights into how its done in practice. In particular: What kind of data are you dealing with? How are you dealing with hyper-param optimization? ...
0
votes
0answers
13 views

Computing the Hessian Matrix Diagonal of a multi-layered Feed Forward Neural Network

I am working on using a Feedforward multi-layered perceptron as a function approximator for the pressure distribution of a groundwater system. I am essentially trying to solve a boundary value problem ...
1
vote
1answer
23 views

Does high training accuracy for an NN mean that it has a potential to reach high validation accuracy?

I saw quite a few discussions related to the problem of high training accuracy with low validation accuracy and what steps to take to address it. I have the same problem with a binary classification ...
1
vote
1answer
27 views

U-Net convolutional neural network

I am currently trying to understand how exactly the U-Net (https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/) works and so far failed to understand some key points, which are the following:...
0
votes
0answers
37 views

Can feedforward NN handle discontinuous functions?

Hi I am trying to simulate the flow of water through a porous medium using ANNs. I have managed to get good result when the porous medium is homogeneous, however when it isn't the network seems to ...
3
votes
1answer
31 views

A Logistic Regression with Neural Network mindset VS a shallow Neural Network

I am a new student in the world of deep learning and after studying the functioning of logistic regression and neural networks there are some insights that probably escape me. Given these two ...
-1
votes
0answers
18 views

What is the reason for developing batch normalization? [closed]

I am reading Deep Learning book by Ian Goodfellow et al., (section 8.7.1) in which they described the difficulty in training deep neural nets and how batch normalization overcomes it. They describe it ...
0
votes
1answer
10 views

When model complexity goes up, why test error also goes up, instead of staying on a similar level?

When model complexity goes up, why test error also goes up, instead of staying on a similar level ? I feel this is countering the intuition that when you add random parameters to a model, it should ...
1
vote
1answer
29 views

optimal subset / joint distribution prediction with machine learning

How can I find the optimal subset of classes for a given entity? For context, say that we have some customers and data about these customers transactions, and a set of possible products to advertise ...
3
votes
1answer
45 views

What can Deep Neural Networks do that Support Vector Machines can't?

When I started studying machine learning in 2002, Neural Networks were on their way out and Support Vector Machines were becoming more and more popular. At the time my understating was that SVM could ...
1
vote
1answer
19 views

Feed Forward Layers - FC -> Relu -> FC, What the idea of using them

I saw in some papers (like “Attention is all you need”) a block called: “Feed Forward Layer” or “Feed Forward Network”. This is a simple block that contains FC -> Relu -> FC , and the main idea is ...
0
votes
0answers
14 views

At what time step does all information become available to an input in a BRNN?

If a bidirectional RNN (BRNN) is essentially two independent RNNs put together, with the input sequence being fed in normal time order for one and in reverse time order for the other, wouldn't some of ...
-1
votes
0answers
26 views

Why can RNNs handle variable length inputs?

Title My understanding is that theoretically, an unrolled RNN is simply several NNs that feed their hidden layer results to the next NN along with the input at time t, therefore, each input can be of ...
0
votes
0answers
21 views

How to build CNN to detect open defects? [closed]

I'm having enough dataset to train a CNN from scratch, but as I'm new to deep learning I'm confused that how I configure a CNN to detect open defects in image, because in dataset there are images with ...
0
votes
0answers
14 views

How does Gaussian prior on weights guarantees that the units are not likely to interact with each other?

In the Deep Learning book [1], Section 8.4, the authors wrote that ... (imposing a gaussian prior on weights) says that it is more likely that units do not interact with each other than that they ...
3
votes
0answers
28 views

Why does Nesterov momentum not improve the rate of convergence in the stochastic gradient case?

I have been reading Deep Learning book by Ian Goodfellow, where they wrote in chapter 8 (section 8.3.3) that Nesterov momentum does not improve the rate of convergence in stochastic gradient case. ...
4
votes
1answer
24 views

Understanding Q-learning for continuous actions

I am reading the paper on Normalized Advantage Functions for continuous Q-learning and I am having trouble understanding why the advantage function takes this particular form: Why is the Advantage ...
0
votes
1answer
36 views
1
vote
1answer
34 views

Why can't we use backpropagation and gradient descent on a Restricted Boltzmann Machine

Can someone please explain why we cannot use the backpropagation algorithm and gradient descent to train a Restricted Boltzmann Machine. In other words, why can't we train an RBM in the same manner ...
0
votes
1answer
23 views

Meaning of different softmax notations in papers

I was wondering if the different notations of the softmax input mean different things especially about the size of the output. For example, in the paper Pointer Networks, it sometimes state the input ...
0
votes
0answers
11 views

What the operation between X(t) and h(t-1) in a Recurrent Neural Network?

Here the state of older time step and input of current time step are used to predict the current state. How are they combined? Is it concatenation? If its concatenation then tanh wouldn't change its ...
-1
votes
1answer
29 views

Stochastic gradient descent and asymptotic analysis

In 8th chapter of deep learning book, the following lines are written under Stochastic gradient descent heading: The asymptotic analysis obscures many advantages that stochastic gradient descent ...
0
votes
1answer
29 views

Does regularization leads to stucking in local minima?

I frequently hear some very conflicting claims regarding deep learning algorithms. Currently, I am a bit confused on the role of regularization. I have listed my queries below regarding regularization ...
1
vote
1answer
32 views

Which algorithm for classification problem?

I want to create a ML (DL) model, that predicts the success of Facebook page-posts, based on historical data. My dataset represents a couple thousands posts, labeled 1 (successful) and 0 (...
0
votes
0answers
16 views

How to train a neural network after the hyperparameters have been found? What is the stopping criteria?

Say for example i use validation score for early stopping to prevent the network from overfitting and find the best hyperparameters using the CV techniques. Now that i have found the best ...
1
vote
0answers
43 views

Explanation of overshooting accumulated reward in reinforcement learning

We are curious about the trend of the total accumulated reward in RL applications. Especially when it comes to an overshooting in the signal at the beginning of the training as shown in the plots from ...