Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

An area of machine learning concerned with learning hierarchical representations of the data, mainly done with deep neural networks.

0
votes
0answers
4 views

What's the relationship between overfitting and network depth?

Intuitively, given a neural network with a fixed number of parameters, as the network grows deeper, it can learn a richer structure and has a bigger hypothesis space. But deeper networks now often ...
1
vote
0answers
14 views

Penalize False negative in Deep Learning

I am doing pathology classification (normal vs different pathologies). I want to penalize more False negative than False positive in convolutional network (with Keras or Tensorflow package). Accuracy ...
0
votes
0answers
3 views

Very high precision and specificity but low recall (sensitivity)

I am working on a project to classify lung CT images into cancer/no-cancer using CNN, the dataset is 13500 samples, 1500 samples are positive (1) with two time augmentation (rotate 60 and 180) its ...
0
votes
0answers
4 views

Metric for evaluating predicted bounding boxes from semantic segmentation on an object level outside of training

Context For simplicity let us pretend we are performing semantic segmentation on a series of one pixel high images of width w with three channels (r, g, b) with n label classes. In other words, a ...
0
votes
0answers
7 views

Is there any papers about learned image features in an unsupervised way?

I know that image features produced by pretrained models like VGG-net are the common way to initialize other networks in other tasks. But I was wondering, Is there image features that have been ...
2
votes
1answer
18 views

What would happen if CNN reused same kernel weights for each channel?

In a CNN each output location of a feature map is given by the kernel over the previous layer's feature maps. If the receptive field is say 5x5 with 3 channels then there are 5x5x3 weights that are ...
0
votes
0answers
21 views

Zero inflated binomial for excessive zeros

"What are some tricks for dealing with a zero inflated response variable when tackling a machine learning regression problem?" Answer : "One of the easiest and most intuitive methods is to run a ...
0
votes
1answer
12 views

Language model in deep learning - hard time to understand the task

I'm having a hard time to understand the task of "language model". Translate, speech, Spelling, sentiment analysis, those I understand, but what does "language model" means?! is it just the action of ...
0
votes
0answers
6 views

Negative log loss messing up training

Hey I have a very simple NN with only one layer. I was going through someones tutorial and they did not take the log in the negative log likely hood loss. This is on the MNIST flattened dataset. When ...
0
votes
0answers
7 views

Why the loss function of Vanilla Policy Gradient cannot tell how good the model is?

When reading Spinning Up (the gray box titled "You Should Know" in the middle of the page), it says the loss function does not indicate how well the model does. I cannot figure out why. I think the ...
0
votes
0answers
4 views

Mixture of experts - Balancing the load between the experts

for the last few weeks I'm learning MoE (Article and code) and there is one thing, I have a really hard time to understand, Balancing the load between the experts, I'll explain using transformer: Let'...
0
votes
0answers
23 views

100% classification accuracy on validation dataset

I am trying to perform a multi-class classification where the network is trained to classify objects into 3 categories: cars, pedestrians and miscellaneous. I am using the KITTI Dataset for car ...
0
votes
2answers
37 views

How to train a stock trading neural network so that the 'profit' parameter is maximized?

I am watching some beginner level video training on neural networks using Tensorflow / Keras to get a better understanding of how they work and how to best implement them. I have some questions on ...
0
votes
1answer
10 views

What is the definition of layer in neural network?

What is the precise definition of layer in neural network? Are things like concatenate functions, activations, batch normalizations, skip connections considered as layers?
0
votes
0answers
13 views

What does anchors' scales actually refer to in Faster RCNN?

I am trying to understand the faster RCNN but I can't understand the meaning of anchors' scales? Especially in this article Faster RCNN. The author considers 3 scales $(128^2, 256^2, 512^2)$ ...
0
votes
0answers
56 views

How can I combine a particular loss function into a DNN with another loss/objective?

I'm training a fully-connected layer with a custom loss $L_1$ to perform dimensionality reduction. This loss is in function of the weights, which pushes the network to a solution which has some ...
0
votes
0answers
9 views

Is softmax a fully connected layer? [on hold]

I know that in all CNN networks there usually a softmax layer at the end of the network. My question is the that softmax layer a fully connected layer or not?
1
vote
0answers
5 views

Storage and re-computation of Intermediate / Weight / Back-propagated Gradients

I need to track the computation and storage of different parts of my network training. To be on the same page, let's assume the simple following scenario (biases omitted) Questions Local Gradients - ...
2
votes
1answer
24 views

Backpropagation wrong? Doesn't it update dependent variables in hidden layer

In a multi layer perceptron or feedforward neural network, isn't backpropagation updating weights of the middle layers that are dependent variables? So for a particular hidden layer, it calculates all ...
0
votes
1answer
23 views

When to use recurrent neural networks?

In order to create a training set for RNNs one typically takes a sequence and turns it into a training set using the sliding window approach. For example if the sequence is: ...
0
votes
0answers
16 views

MNIST database with european-style handwritten digits [on hold]

The MNIST database is useful to train a digit-recognition neural network. However: most of the "9" are written without the "lower curve" of the 9, whereas in France (Europe?) most people write it ...
1
vote
0answers
10 views

How to take advantage of additional data?

I am trying to train a 3 class problem object detection problem,For these 3 classes i have around 9000 samples each.The model is performing decently but there is still confusion between classes. I ...
1
vote
2answers
33 views

Deep learning methods with seasonal data

I have been building an LSTM model in Python which will predict the number of passengers arriving at a station in the next 15 mins. My dataset is arrivals at the station every 15 mins across a 50 day ...
0
votes
0answers
9 views

Back-propagated gradients vs Weight gradients? [on hold]

I am trying to reproduce “Understanding the difficulty of training deep ffnn” using PyTorch and extend the same analysis to more scenarios. I have both theoretical and practical questions regarding ...
0
votes
0answers
23 views

Joint probability distribution of correlated data points

I have a query with respect to joint distributions. Here, each output data point in $\mathbf{y}$ is conditionally independent given the inputs $\mathbf{x}$ and the mapping $f:\mathbf{x}\rightarrow \...
1
vote
1answer
17 views

How to pre-train a deep neural network (or RNN) with unlabeled data?

Recently, I was asked about how to pre-train a deep neural network with unlabeled data, meaning, instead of initializing the model weight with small random numbers, we set initial weight from a ...
0
votes
0answers
10 views

Speech recognition - hangover scheme - voice activity detection

I am doing a voice activity detection challenge, and I am asked to add a hangover scheme to the model. I read about hangover schemes in different papers but I couldn't find a definition for this. What ...
2
votes
2answers
612 views

Asymmetric cost function in neural networks

I am trying to build a deep neural network based on asymmetric loss functions that penalizes underestimation of a time series. Preferably, by the use of the LINEX loss function (Varian 1975): $ \...
0
votes
3answers
103 views

Dealing with excessive number of zeros

ipdb> np.count_nonzero(test==0) / len(ytrue) * 100 76.44815766923736 ...
0
votes
2answers
44 views

Why is LSTM so ineffective at this ridiculously simple sequence?

I am trying to predict the the sequence $(0.1, 0.05, …, 1.1)$ from the sequence $(0, 0.05, 0.1, …, 1)$. I thought this would be the only item in a toy dataset. Now I implemented a long-short term ...
0
votes
0answers
10 views

Different random weight initilization leading to different performances

I'm training a 3D U-Net on an EM dataset of a brain. The objective is to segment neurons in it. During the experiments, I've noticed, different random initialization of the network leads to different ...
0
votes
0answers
10 views

Batch Normalization and increasing batch size reducing the performance

I'm training a 3D U-Net on an EM dataset of a brain. The objective is to segment neurons in it. During the experiments, I've noticed, increasing batch size, adding batch normalization layers (Conv ->...
0
votes
0answers
14 views

Agents playing against each other (Reinforcement Learning)

To train my reinforcement model to play a two player game I could either play by myself or let it play against a second instance of itself. Are there any drawbacks which I have to take care of? Some ...
3
votes
2answers
25 views

Encoder Decoder networks with varying image sizes

Encoder Decoder Network - Computerphile : At the very beginning of this video, Michael Pound goes on to say: So it (encoder decoder network) makes no assumptions about the size of the input the ...
1
vote
1answer
17 views

RMSprop difference rho and decay in Tensorflow

As this post showed nicely, there is a difference between rho and the decay in RMSprop. I can't clearly see what tensorflows RMSprop parameter decays stands for. Is this the learning rate decay? And ...
0
votes
0answers
7 views

strange loss trajectory for vgg11

I trained a vgg network with 11 layers from scratch and am observing some strange behavior with the loss plot (see below). The loss is binary cross entropy. My learning rate is 1e-4 and I decreased ...
0
votes
0answers
17 views

using (deep) neural networks for a severely imbalanced image dataset when some classes have <10 images

Taking a long shot here. So I have a a small dataset of ~500 images with discrete labels from 1 to 9. My task is to detect the per-class and overall accuracy of this classification method using a (...
0
votes
2answers
26 views

convolutional autoencoder on an odd size image

I am trying to apply convolutional autoencdeor on a odd size image. Below is the code: ...
0
votes
1answer
51 views

MNIST digit recognition: what is the best we can get with a fully connected NN only? (no CNN)

To fully understand how it works internally, I'm re-writing a neural network from scratch in Python + numpy only. (As it's for learning purposes, performance is not an issue). Before moving to ...
0
votes
0answers
16 views

Query on LSTM (Request for guidance)

I recently started working on a project at the University. The main task of the project is to apply Deep Learning for forecasting. I have the dataset from a company that basically contains various ...
0
votes
0answers
15 views

Training a multi inputs deep learning model using every combination of inputs?

I am beginner in deep learning. I want to create a multi inputs CNN model in Keras. The model takes two inputs of images to give the two images class. The two images from differnt datasets that have ...
1
vote
1answer
17 views

Why map the pixel grayscale [0, 1] to [0.01, 0.99] before feeding to the neural network? (MNIST digit recognition)

In this introduction to neural networks (I enjoy it because it builds a digit-recognition neural network from scratch with just numpy, without any high-level NN library like pytorch or tensorflow; ...
0
votes
0answers
14 views

U-Net image size for training

I have a small question regarding the size of images used for training the U-Net. I have thus far been able to train a U-Net reasonably well using 656x656 images and now wanted to use sections of ...
0
votes
1answer
26 views

My deep learning model works with Stratified K-Fold, but not in predicting future time events

I have a project at work where I'm supposed to predict a specific location given cell phone signal parameters. I developed a deep learning model that uses convolutional neural network on time-ordered ...
0
votes
0answers
8 views

Demand forecasting for sequence data and multi-task learning

I have a demand forecasting problem that I'd like to solve with a deep learning using multi-task learning and I'd like advice in some areas. Problem definition: I have a set of $N$ customers that ...
0
votes
0answers
11 views

Unsupervised learning with DNN on text

I want to: Have a key:pair database with author:largetextfileofeverysentencetheauthorpublished.txt Set up a deep neural network to see without supervision patterns in choice of vocabulary. Have the ...
1
vote
0answers
4 views

Scene annotation

I am looking to run my convolutional net on a tennis video and see if it can recognise the type of shot played( straight, volley or cross court). What tool can I use to annote the scenes in my video?
0
votes
0answers
24 views

keras embedding training optimization objective

I am aware of this and this existing questions, as well as this issue on github. Unless I am missing something though, all these fail to explain how the example in the keras docs makes sense: ...
0
votes
0answers
17 views

Is there any theoretical prove that Logistic Regression has better probabilistic interpretation than Deep Neural Networks?

I've heard that Logistic Regression output distribution has good probabilistic interpretation. Also, I can easily see that in many problems DNNs are overconfident and bad-calibrated. Are there any ...
1
vote
0answers
30 views

Recognition the same object from different views

I have 33 classes (33 different objects). I need to recognize the object from any view of the object. Like a packet of potato chips, the packet has different appearance from different view (as shown ...