Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

An area of machine learning concerned with learning hierarchical representations of the data, mainly done with deep neural networks.

0
votes
0answers
8 views

Neural network how to deal with comparison

I'm currently working on a DQN network and this question comes to me. As far as I know, neural networks are good at dealing with values that have never seen (generalisation). E.g. If a classification ...
0
votes
0answers
8 views

How to compare the training performance of a deep learning model on different data sets?

So I have a deep learning model and three data sets (images). My theory is that one of these data sets should function better when it comes to training a deep learning model (meaning that the model ...
0
votes
0answers
12 views

CNN and kernel sizes: is upsampling useful?

I am playing with Deep Recurrent Q-Network in Reinforcement learning. The architecture I am currently using is similar to the one presented in "Human-level control through deep reinforcement learning"...
2
votes
0answers
23 views

Similarity between Train and Test data sets

I have multiclass classification dataset and I am using Deep nets for the classification task. To explain the problem, let's assume that I have 5 classes to classify. No matter what I try, be it ...
0
votes
0answers
38 views

Algorithms for everyone [on hold]

Something I've allways wanted to see is a concise run-through of different machine learning algorithms, all on one page: With their pros and cons, what situations they work in best and when they don'...
1
vote
1answer
23 views

Help about cosine simulating in h2o

So I'm beginning in deep learning and especially in h2o. I tried to simulate cosine function in R, not to compute it like for example by using h2o.cos(), But after many and many more combinations of ...
0
votes
0answers
6 views

balance_classes in H2O, but for regression?

I am training with deep learning for regression in H2O for R. My dataset is unbalanced (ie. not evenly distributed). There has been discussion on whether unbalanced datasets are an issue or not, with ...
0
votes
0answers
8 views

Computation complexity and processing of one image for object detection in Convolutional Neural Network

How do I relate compute complexity in Convolutional Neural Network to processing time of one image in object detection for a given CPU/GPU's processing power? Say my CNN architecture needs ...
0
votes
1answer
31 views

How can a network with only ReLU nodes output negative values?

I'm trying to use an api with a feedforward neural network for time series forecasting. For dense aggregate data it works fine, but for sparse data it sometimes forecasts negative values, even though ...
1
vote
1answer
25 views

Why ensemble of many deep-learning models did not work?

I am trying to solve an image classification problem using DL, Keras and tensorflow. I added several layers of conv2D followed by batchnorm, pooling and dropout. I get a good accuracy ~95% with this. ...
1
vote
2answers
15 views

Is it better to avoid ReLu as activation function if input data has plenty of negative values?

ReLu is probably the most popular activation function in machine learning today. Yet, ReLu function outputs 0 when input data values are negative. ReLu totally disregards negative data. This may ...
9
votes
3answers
192 views
+150

What *is* an Artificial Neural Network?

As we delve into Neural Networks literature, we get to identify other methods with neuromorphic topologies ("Neural-Network"-like architectures). And I'm not talking about the Universal Approximation ...
1
vote
1answer
15 views

RNN vs Convolution 1D

Intuitively, are both RNN and 1D conv nets more or less the same? I mean the input shape for both are 3-D tensors, with the shape of RNN being ( batch, timesteps, features) and the shape of 1D conv ...
-3
votes
0answers
19 views

Separate images into training, validation and testing in Keras other than do it manually? [on hold]

I want to use deep learning to train a dataset that has more than 100 classes. I want to sperate the dataset into three sets(train, test, validate), but it is exhausted to sperate them manually in ...
0
votes
1answer
16 views

Dimensionality Reduction on VGG Image Vector

I have a random forest model which I am using to make retail demand predictions. I am looking at trying to leverage product image data to improve the predictions and have put the images through VGG-...
0
votes
1answer
17 views

Reduction of Feature map size in Convolutional Neural Network

In CNN, the way we reduce the feature map size at layers is we use pooling. Pooling makes feature map size into half. For the following network, if I want a new layer with feature map size somewhere ...
1
vote
0answers
150 views

Model for predicting chance of winning in variable count of opponents

I have dataset with horse racing results including bookie odds - converted to percentage chance of winning. Data are stored in relation tables. The basic entity relation is described on image. Each ...
0
votes
1answer
19 views

Cross Entropy calculation question: calculated is different from Keras' output

I wrote a simple code to test Keras cross entropy, but got different results from this post. I checked everything, but still do not know why keras gives me ...
1
vote
1answer
73 views

Time-series prediction with RNNs: What to expect from the learning process?

When training an RNN for time series prediction, what can one expect to see visually as the model learns? In particular, are plateaus a normal indication that the model is underfitting or do they ...
1
vote
0answers
11 views

Deep learning unerfitting/overfitting

I started to perform deep learning for sentiment analysis on word embedding. I have plot the model loss and accuracy graph for each epochs to understand the performance better. I read the following ...
0
votes
0answers
26 views

RNN not learning Keras [closed]

I have been training RNNs in keras for some time now, but recently I was faced with a problem of RNN not learning anything. Therefore, I want to make sure I am feeding the data correctly. I know that ...
1
vote
0answers
22 views

Adverserial Verification of an XGboost Classifier

This paper proposes an algorithmic framework, predictor-verifier training, to train neural networks that are verifiable, i.e., networks that provably satisfy some desired input-output properties. The ...
0
votes
0answers
26 views

prioritized experience replay (PER)

I have a question about prioritized experience replay (PER) proposed as a an improvement for DQN here. As it has been mentioned there, here to take k samples, we break the interval of (0, sum of ...
2
votes
1answer
24 views

Extensions of LSTM for huge data

Consider dealing with a huge high frequency financial data forecasting, RNN/LSTM is a popular way to solve the task. But the problem is that say you have 1 million data points and you want to predict ...
4
votes
0answers
26 views

Weight normalization technique used in Image Style Transfer

I am trying to implement the paper Image Style Transfer Using Convolutional Neural Networks. In section 2 - Deep image representations, the authors mention the following weight normalization technique:...
1
vote
0answers
9 views

I am trying to build a progressive auto encoder neural network and I am not sure how to discard old weights?

The goal of the network is simple, encode and decode images at a smaller scale and slowly increasing the network complexity, the input image size and its output quality. My current weights for my ...
1
vote
1answer
52 views

Active Learning with Human-in-the-Loop

I did a lot of research and can't find a satisfactory answer. I have just a quick question about Active Learning and would be pleased if you could answer it. I'm still wondering if active learning ...
1
vote
1answer
23 views

When should I stop a training of WGAN model?

The loss function of the WGAN is a continuous one. It doesn't have a convergence point. I don't really understand when we should stop the training.
1
vote
0answers
15 views

Adaptive Moment Estimate - What is meant by parameters of Adam optimizer are biased towards zero initially?

Let us consider the Adam optimizer with the equation given below: $w_{t + 1} = w_{t} - \frac{\eta \times \bar{m_t}} {\sqrt{\bar{v_t} + \epsilon}}$ Here $w$ denotes weight (in time $t$ and $t + 1$) ...
0
votes
0answers
20 views

Oversampling for multi-class neural net

Does this make sense or do I have no idea what I'm doing? I want to train a model that takes a sentence and outputs a binary multi-class vector of size $K$ where each dimension is a question class. ...
0
votes
0answers
10 views

Data augmentation or weighted loss function for imbalanced classes?

I have a CNN image classification problem with imbalanced classes. I could balance the dataset using data augmentation (Replication, mirror, etc.) on the minority classes. Also, imbalance could be ...
2
votes
0answers
22 views

Binary vs Multiclass Accuracy

Consider an image dataset that has two types of things: cars and airplanes. Let $A$ be a binary classifier trained to classify an image as having a car or airplane. Suppose we now have four refined ...
2
votes
0answers
36 views

Assumptions for RNNs

Machine learning doesn't put emphasis on the assumptions for the data generating process, but on prediction. However, some assumptions must be made so that the trained model can be applied to new, ...
0
votes
0answers
13 views

Beam search in seq2seq decoder

If i use a beam search decoder, say of length 2. I have ['Hi','Bye'] as current two best options. Now as i move to the next word, i have ['eos','Adam'] as the next two best options. Now here should '...
0
votes
0answers
23 views

Use Deep learning or RL to maximize long term hit in caching file?

Suppose I have F video files with different file size and with different popularity. I have a memory cache of fixed size. What I have to do is following: I want to cache n files (n < F) in ...
2
votes
0answers
27 views

Can few-shot semantic segmentation help improve accuracy for the minority classes?

Few shot learning (Or one shot learning) for the image classification problem can be used when there are few samples per class in the dataset (One method is siamese networks). Few shot semantic ...
0
votes
1answer
24 views

Is there any sense to train deep conv net from the scratch after dataset changes?

I train deep conv model called resnet50 as the object detector. Time to time I make some changes in dataset or data augmentation. I usually use my last checkpoint to continue training on whole changed ...
2
votes
4answers
42 views

Difference between strided and non-strided convolution

conv = conv_2d (strides=) I want to know how non-strided convolution differs from strided i know how convolutions with strides work but don't know about the non-...
1
vote
0answers
25 views

Parameter tuning trick in Reinforcement Learning

In normal deep learning, there requires a lot of training tricks such as inserting a normalization after maxpooling layer or augment the input image by swirling the original input. However, ...
1
vote
0answers
11 views

different between effect of episodes and time in DQN and where is the updating the experience replay

In DQN paper of DeepMind company, there are two loops one for episodes and one for running time in each step (one for training and one for different time-step of running). Am I right? Since, nothing ...
6
votes
1answer
632 views

Should I gloss over the linear algebra chapter in the book “Deep Learning” by Ian Goodfellow?

Currently I am reading "Deep Learning" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville. I'm on Chapter 2 which is the Linear Algebra section where they go over the linear algebra that pertains ...
1
vote
1answer
34 views

seq2seq in pytorch [closed]

I have an encoder LSTM whose last hidden state feeds to the decoder LSTM. If i call backward on the loss for the decoder lstm, will the gradients propagate all the way back into the encoder as well, ...
1
vote
1answer
34 views

Why is dropout causing my network to overfit so badly? [closed]

I've been experimenting with various simple neural networks to test their performance. When I use the following architecture, I'm getting some very bad test error, which looks like overfitting. $$\...
0
votes
0answers
22 views

Which features to extract from circular shape like blobs for classification of objects

I have circular like objects in raw images (first image attached) and there are three object size of these similar shapes. I want to classify them as small-size, medium-size and large-size using ...
0
votes
0answers
11 views

training GANs to inpaint images

I want to train a GAN, the PGGAN from NVIDIA(official implementation available with Tensorflow here), to inpaint images which have been cropped in free form. I have a data set of images from a ...
2
votes
1answer
21 views

Can i use deep learning to measure the similarity between two variable length voice sequences?

Usually we use DTW(Dynamic Time warpping) to measure the similarity between two variabel voice sequences. However DTW is time-cunsuming and not easy to run in the GPU since too much control in it. I ...
3
votes
1answer
32 views

How to compare two different algorithms for deep RL?

Assume we have two algorithms for a deep RL problem. We used them to train an agent for this environment and stored the rewards at each episode for these two different algorithms. My question is that ...
0
votes
0answers
24 views

Intuition behind mean reward\return plots in deep RL

My question is about a common plot which is used in deep RL papers. In these papers, the mean of agent's reward/retrun is plotted versus number of episodes (something like figure below). I wanted to ...
0
votes
1answer
25 views

Epoch Vs Iteration in CNN training

There are a few discussions for Epoch Vs Iteration. Iteration is one time processing for forward and backward for a batch of images (say one batch is defined as 16, then 16 images are processed in ...
3
votes
1answer
42 views

How initializing weights to a certain variance reduce vanishing / exploding gradients? [closed]

Let's assume we have a network with 4 features and one neuron: Accroding to the course "weight-initialization-for-deep-networks" with Andrew Ng, we want to set the variance of the weights to $Var(w)=...