Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [deep-learning]

An area of machine learning concerned with learning hierarchical representations of the data, mainly done with deep neural networks.

0
votes
0answers
6 views

What kind of impact do autoencoders have on final model performance when compared to models trained only on supervised data?

For example, say we have two datasets, a labeled set (I will call it df_labeled) of nrows=200k and an unlabeled dataset (df_unlabeled) of nrows=800k and we want to build a binary classifier. I clearly,...
0
votes
0answers
18 views

What does the error of the neural network model mean? [on hold]

I m fiting a neural network model using R and with the library(neuralnet) but i found the error of the neural network model 500.222 that is not logical at all .. I got this Error when i wrote this ...
0
votes
0answers
15 views

Can a trained neural network recognize rotating characters? [duplicate]

Suppose I have a trained neural network that can recognize, for example numbers from 1 to 10, the size of the picture $28 \times 28$. I made the rotation of these pictures by 90 degrees. Does now ...
0
votes
0answers
13 views

pytorch : seeking explanation for model.forward function

I am learning deep learning and am trying to understand the pytorch code given below. I'm struggling to understand how the probability calculation works. Can somehow break it down in lay-man terms. ...
1
vote
1answer
31 views

Derivative of the loss function w.r.t to X for the backpropagation

I would like to ask you why do we need to calculate a derivative of the loss function w.r.t X? It seems like, that for the backpropagation we need to calculate only a derivative w.r.t W. Can you ...
2
votes
0answers
18 views

Validation ROC AUC not improving with validation cross-entropy loss?

I am training a neural network that is doing binary image classification on several thousand images. I am running 5 fold cross validation (train on 4, validate on 1) with cross entropy (CE) loss. I am ...
1
vote
2answers
17 views

Pooling vs. stride for downsampling

Pooling and stride both can be used to downsample the image. Let's say we have an image of 4x4, like below and a filter of 2x2. Then how do we decide whether to use (2x2 pooling) vs. (stride of 2)?
0
votes
0answers
15 views

Why is number of convolution filters usually powers of two? What's good for that?

I'm studying deep learning these days, I'm a..newbie I guess lol I noticed that there are many "powers of two" in lots of places.. For example, number of convolution filter, batch size etc. I'm ...
0
votes
0answers
8 views

What are the constituents of “distributions” in GANs?

We have a distribution for the Generator and the Discriminator, and we minimize their divergence, but how do the inputs (say, images) constitute a probability distribution? Or is the distribution ...
1
vote
1answer
14 views

On masked multi-head attention and layer normalization in transformer model

I came to read Attention is All you Need by Vaswani. There two questions came up to me: 1. How is it possible to mask out illegal connections in decoder multi-head attention? It says by setting ...
0
votes
0answers
8 views

Which GAN is the best for data augmentation?

I have around 200000 images and I want to augment the data by generating more of them. Images do not have classes, because they are the same object and are used for the task of object detection. Can I ...
1
vote
0answers
47 views

Hottest news on 'why does Deep Learning work so well' [closed]

At the beginning of last year I was trying to study some papers which were tackling the question "Why does deep learning work so well", but I had to stop due to overwhelming work problems. I'd love ...
1
vote
0answers
22 views

Could machine learning be used to select best parameter for more than one loop optimization?

I do not know if i can ask this question here or not. But I really need it. I'm very new to ML/DL/NN field. I have seen many articles tackling the problem of selection of the best parameter for loop ...
0
votes
0answers
7 views

how to separately optimize the parameters of convolutional layers and the parameters of fully connected layers [closed]

I want to use two optimizers to optimize the parameters of the CNN (convolutional layers and fully connected layers) in ...
-1
votes
0answers
9 views

Where should i start to deal with language processing [closed]

what should i need to implement in order to process translation simultanously from one language to english in NLP python
0
votes
1answer
15 views

Data augmentation methods for Raman Spectra

I'm building a CNN model based on Raman spectroscopy data and I wanted to experiment with data augmentation. What would be some reasonable techniques to try? I have found this paper which suggests ...
1
vote
1answer
27 views

How much data is needed to train CNN from scratch?

Any rule of thumb, on how many input images would be needed to have a reasonable chance not to overfit the data when training a CNN from scratch? In other words, what is a reasonable amount of data (...
0
votes
1answer
19 views

Why can't we use back propagation in “Hard attention” but we can use it in “RELU” function and max-pooling?

RELU, argmax function(in hard attention) and max-pooling are non-differentiable functions but We use back-propagation with RELU and max-pooling without any problems. What does make "Hard attention" ...
1
vote
1answer
26 views

Strange batch loss in keras

Im training a Bidirectional RNN with keras.losses.MSE and have my dataset shuffled before training. I manually split it into validation and train data. However when ...
1
vote
1answer
33 views

Future of statistical methods in image segmentation? [on hold]

I was looking for a purely statistical method for image segmentation and found many, e.g. Hidden Markov Random Fields with EM algorithm. But it seems to me that these methods are nowadays completely ...
0
votes
0answers
11 views

What's the difference between random and deterministic encoder in autoencoders?

I read this paper "Wasserstein Auto-Encoders", and they mention deterministic encoder and random encoder but without stating the difference between them. How can we tell the difference?
0
votes
1answer
22 views

Classifying XOR grid with simple NN, but with more points on the grid

I am getting started with very simple Neural Networks/Multilayer Perceptrons. I successfully classified the XOR problem, but I wanted to explore so I created a grid such as . I used Tensorflow code ...
1
vote
1answer
22 views

Data Augmentation in Keras: How many training observations do I end up with?

I'm reading through Francois Chollet's "Deep Learning with Python" and was recently introduced to a concept I had never encountered before in my statistics studies. Namely, data augmentation. I have a ...
1
vote
1answer
30 views

Fitting model on whole dataset, more or less epochs ? (w.r.t validation accuracy)

When tuning my neural networks hyperparameters I use 20% of the data set as validation data. With the holdout set I observe the validation accuracy and validation loss. In my case the model starts ...
0
votes
0answers
24 views

Autoencoder as an optimization (search) problem

We all know that machine learning problems can be modeled as an optimization problem where we are searching for the best set of parameter values in the parameter space that optimizes our objective ...
0
votes
1answer
36 views

What does decay_steps mean in Tensorflow tf.train.exponential_decay?

I am trying to implement an exponential learning rate decay with the Adam optimizer for a LSTM. I do not want the 'staircase = true' version. The decay_steps for me feels like the number of steps that ...
0
votes
2answers
24 views

Keras model optimization of 2D arrays

I am trying to train a CNN with 2D arrays of normalized numbers. Example of 2D training array: ...
0
votes
0answers
24 views

Deal with data from spectrometry

I'm trying to predict type of cells (A or B) using data from a spectrometry, here is the shape of data I have : I'm facing the following problems : each value of the spectre is represented by a ...
0
votes
0answers
13 views

Types of preprocessing for Deep Learning NLP tasks

I am doing some research with Deep Learning NLP tasks. There are many ways of text preprocessing. Some are removing stop words. Others convert to lower case, do stemming, or lemmazation. Others do ...
0
votes
0answers
16 views

How do convolutional neural networks deal with many filters during convolution?

I am unsure of how convolutional neural networks treat several filters. Many of the examples I have seen only have filter at a time, and that is intuitive for me. Look at the nice visual tutorial here:...
0
votes
0answers
28 views

Log likelihood function for (neural networks) regression

My question is about how we calculate the loglikelihood function for regression when you have multiple standard deviations instead of a single standard deviation. For a standard linear regression (...
1
vote
0answers
21 views

Initialize replay memory and action value function Q

I am not sure I can ask that question here, but I will try an attempt. I am trying to implement Beat Atari with Deep Reinforcement Learning. They explained very well each steps, but they ask you to ...
0
votes
0answers
25 views

Why does gradient descent work faster with ReLU compared to using with Signoid? [duplicate]

As far as I understand, Signoid function is used for mapping the outputs of neural network to the values between 0 and 1. Why is using rectified linear unit(ReLU) as activation function in deep neural ...
1
vote
0answers
14 views

How does Inspirobot random insprational quote generator work

Inspirobot is a website that generates random inspirational quotes. I would like to understand how this was built (training data used, algorithms used to create the sentences, etc). Please reference ...
1
vote
1answer
45 views

sample data for training neural networks for self-driving cars [closed]

If I ask the question in the wrong forum, let me know, I will delete it. I want see sample data for training neural networks for self-driving cars. I understand that there will be geodata and image ...
0
votes
1answer
35 views

How to apply multi agent deep reinforcement learning to an environment with discrete action space

Do you know or have heard about any cutting edge deep reinforcement-learning algorithm which can be successfully applied for discrete action-spaces in multi-agent settings? I have been researching ...
2
votes
1answer
39 views

Why increasing the batch size has the same effect as decaying the learning rate?

There have been a few papers this year, concerned with very large scale training, where instead than decaying the learning rate $\eta$, the batch size $B$ was increased, usually with the same schedule ...
0
votes
1answer
13 views

How to handle unrelated input such as letters on a trained deep learning MNIST dataset

The MNIST dataset (usually trained with CNN) aims to recognize the following numbers {0,1,2,3,4,5,6,7,8,9}. So it is trained under the assumption that the input should be a number between 0 to 9. ...
0
votes
0answers
23 views

How to encode text and categorical variables together?

I have two groups of texts that are very similar (e.g. reviews written on fridays and reviews written on mondays), and I want to build a LSTM that can classify them into positive and negative reviews. ...
0
votes
1answer
14 views

About RNN with variable length output vectors

I have several thousands samples with equal number of features (5000, they are time dependent) and I would like to predict of vectors with variable length. 1) I'm beginner in RNN, and I'd like to ...
1
vote
0answers
21 views

Deep Learning Variable Length Sequence Handling

I am trying to understand the best practice for handling different lengths of sequences in NLP tasks. Lets consider an example of convolution on sequences followed by max pool layer. We can handle ...
0
votes
0answers
44 views

Neural Network for the Famous Black-Scholes Equation (1972)

The price of an option (in finance) is given by the famous Black-Scholes equation. I would like to design a neural network to predict the price of an option. Basically the inputs are the attributes of ...
0
votes
1answer
38 views

Can I create training split as follows?

I currently have 10000 images for class A and 1000 images for class B. Instead of undersampling or oversampling, I would like to split the class A data into 10 fold and train with available class B ...
0
votes
1answer
22 views

Why we call ADAM an a adaptive learning rate algorithm if the step size is a constant

In the book "Deep Learning" by Goodfellow et.al, the ADAM algorithm is described in sub-chapter 8.5 "Algorithm with Adaptive Learning Rate". To my understanding an adaptive learning rate should ...
2
votes
1answer
25 views

Should training data in each batch size be resample only one time or at each epoch when using mini-batch

I saw some related question regarding to the fact is one should use sampling with resampling or without when using minibatch. However my question is different. Let's assume that I use sampling ...
8
votes
4answers
111 views

Likelihood-free inference - what does it mean?

Recently I have become aware of 'likelihood-free' methods being bandied about in literature. However I am not clear on what it means for an inference or optimization method to be likelihood-free. In ...
0
votes
0answers
21 views

Can existing hyperparameters be used when new features are added to data?

Lets say I have a 1D CNN and a dataset on which I have run bayesian optimiztion and I have the best hyperparameters (decided by lowest loss). Now if I decide to add new features to the data, keep the ...
0
votes
1answer
13 views

Should I be stepping down high a dimensional embedding when predicting low dimensional output

I'm using a ResNet-50 pretrained on ImageNet as a starting point for various image classifiers. Because the pretrained model has 1001 outputs, I have added a single dense layer with output size 500 ...
2
votes
1answer
43 views

Tensorflow batch normalization for images - padding issue

I'm trying to train anomaly/defect detection network on custom images. Let say I have to detect scratches on special steel boxes and I have two views: side view with dimension 2300 x 550 (width x ...
1
vote
1answer
66 views

Why are GANs so innovative?

I've been reading about the importance of Generative Adversarial Networks (GANs), and I would like to double check that I understood correctly why they are so relevant. Before GANs, what people did ...