Questions tagged [neural-networks]

Artificial neural networks (ANNs) are a broad class of computational models loosely based on biological neural networks. They encompass feedforward NNs (including "deep" NNs), convolutional NNs, recurrent NNs, etc.

Filter by
Sorted by
Tagged with
1
vote
0answers
23 views

Handling time series data with seasonality and trend for training LSTMs

Do I need to remove seasonality and trend from time series data, before using it for training lstm. Or, with sufficient data, can LSTMs recognize the pattern in trend and seasonality ? I have 3 years ...
1
vote
0answers
15 views

Bayesian optimization experiment to confirm learning rate schedules in DNNs

Common learning rate schedules usually decrease the learning based on some criteria or some predefined schedule which intuitively makes sense. Has it been confirmed with bayesian hyper parameter ...
1
vote
0answers
9 views

Loss values in a Domain Adversarial

I have been using ResNet-50 with Domain-Adversarial network. I observed an oscillation in the loss values from the evaluation as you can see in the figures, this oscillation was not observed when ...
1
vote
0answers
15 views

Can I frame the same time series forecasting problem either as stateful or stateless lstm

If I have to predict the weekly sales of the product from the past. My data is at weekly level for the product and has about 5 years of data i.e, 260 data points and have about 20 (independent) ...
1
vote
0answers
25 views

Minimum neurons needed to implement any binary function with 1-hidden-layer

I am trying to design a 1-hidden-layer neural network to implement parity bit checker for 5-bit length inputs, wherein each neuron has a simple threshold activation i.e. $$ a(z)= \begin{cases} 1 &...
0
votes
1answer
27 views

convolutional neural network - how filters are found/calculated

I am trying to learn convolutional neural networks from scratch. I try to find a simple example that I can calculate by hand just to get the ideas. There are many things that I do not understand so ...
1
vote
0answers
30 views

Time series prediction completely off using ESN

I am attempting to predict closing prices based on closing prices extracted from OHLC data from a two-month window with 10 minute intervals (roughly 8600 data points). For this attempt, I am building ...
2
votes
1answer
61 views

Why does my model produce unrealistic output?

I am trying to run a binary classification problem on people with diabetes and non-diabetes. For labeling my datasets, I followed a simple rule. If a person has T2DM...
2
votes
2answers
26 views

Why is the default learning rate for Adadelta so low in Keras?

I have been training a model using the Adadelta optimizer for some time, and I noticed that it converges very, very slowly. Then I checked the Keras documentation, and to my surprise the default ...
0
votes
0answers
13 views

HMM or RNN for modeling and prediciting deep sleep

I have created a hidden markov model (HMM) that predicts if a drosophila is awake, in light sleep, or in deep sleep using binary movement data as the input / observable. However, I was wondering if a ...
4
votes
2answers
27 views

Which output activation is recommended when predicting a variable with a lower but not an upper bound?

I need to predict something using a neural network. The output values are bound to be non-negative, but there's not really an upper bound. I do know that the output is never going to be higher than a ...
6
votes
1answer
207 views

On solving ode/pde with Neural Networks

Recently, I watched this video on YouTube on the solution of ode/pde with neural network and it motivated me to write a short code in Keras. Also, I believe the video is referencing this paper found ...
0
votes
0answers
20 views

How to create one hot segmentation masks from rgb mask image for multiclass segmentation?

I am trying to train Deeplabv3 for semantic segmentation on BDD100k dataset. It contains 20 classes for segmentation task. In the Dataset labels are provided as RGB mask image (3 channels). How do I ...
0
votes
0answers
34 views

Why does switching the architecture result in nans during training?

What are my options to figure out the casue of spontaneous nans/infs during training? Basically I changed the model from a custom resnet to the vanilla resnet(and also later a vggnet varation) and ...
2
votes
0answers
30 views

Correlated random variables and ensembles (law of large numbers?)

Consider $n$ i.i.d random variables. By the law of large numbers (LLN) the sample average would converge after some time to the expected value. Let's assume the random variables are correlated. Would ...
0
votes
0answers
15 views

NLP technique for multiple sentence fusion (combination) into one readable sentence

looking for help in knowing if there is a possible solution in natural language processing that could help me use or build a model that combine two or more different sentences into one for example: ...
1
vote
1answer
12 views

Seq2Seq Machine Translation Question

I'm reading through Pytorch's NLP from Scratch: Translation with a Sequence to Sequence Network and Attention, and I am a bit confused on the Preparing Training Data section, particularly: ...
0
votes
1answer
25 views

Is ML and DL turning math into code? beginner

I've gathered some interest over artificial intelligence (ml and dl) and noticed a lot of it is turning math into code if you wish to improve upon a model that will make a prediction. If you wanted to ...
0
votes
0answers
14 views

Refining a Human's Guess - Reinforcement Learning?

I'm having trouble choosing a model for this particular problem. Say in a real world environment, I have different materials I would like to grow until they reach a threshold. Two things affect the ...
4
votes
1answer
261 views

Does Attention Help with standard auto-encoders

I understand the use of attention mechanisms in the encoder-decoder for sequence-to-sequence problem such as a language translator. I am just trying to figure out whether it is possible to use ...
2
votes
1answer
24 views

Small, simple neural network test problem?

I beginning to learn about neural networks and how to train them. I've been reading about using gradient descent for training. Most books go straight to the backpropogation algorithm for computing $\...
0
votes
0answers
12 views

Keras model fit: validation_data vs validation_split [duplicate]

I would like to clear my doubts about these 2 parameters: validation_data and validation_split of the ...
0
votes
0answers
27 views

What is the context between texture features and image anomaly detection?

I have been looking into deep image anomaly detection, more specifically the feature extraction component, for quite a while now. (My focus is on deep learning) I have encountered various papers about ...
2
votes
1answer
63 views

Is there a better way to describe a model's generalization performance than “under” and “overfitting”?

To me, under and overfitting are the two of the most vague concepts in machine learning. From Google's first link when you look up these definitions. A model is said to be underfitted if it "...
3
votes
0answers
34 views

What are the non-differentiable neural network architectures?

Neural networks are generally not differentiable (in the rigorous mathematical sense) due to activation function such as ReLu Recently I have been wondering about the other possible sources of non-...
5
votes
0answers
44 views

Local optima in high-dimensional optimization

I remember a theorem along the lines of In higher dimensional optimization problems, you are less likely to get stuck in local optima, because the more dimensions you have, the more likely you are to ...
2
votes
1answer
28 views

Overview Feature Extraction in images?

I have been searching for deep feature extraction approaches for a while now, but I did not find a single paper giving me a coarse overview on this matter. Apart from an overview, for example I would ...
1
vote
1answer
34 views

merge two neural networks at test time [closed]

I am currently using one network which gives me a bad resolution result, so then I use another network to enhance the resolution of my output. My question is: is there an easy way to use both of them ...
0
votes
0answers
11 views

Which model to use? (cross validation with early stopping)

In this example, to keep things simple we use only 1 training and validation set, and we are trying to find the best regularization parameter for ridge regression. The square loss below is on the ...
0
votes
0answers
27 views

DNN underestimates high values

I'm running a DNN on a dataset in order to predict the output values (y). The actual vs fitted graph shows a slight overestimation of the small values and an underestimation of the higher values. The ...
0
votes
0answers
13 views

What activation and where to use in MaskRCNN RPN

so I've been trying to implement my own version of MaskRCNN, and I am baffled by how the RPN is implemented in various places. Assuming the standard RPN architecture of a shared 3x3 Conv2d, and two ...
0
votes
1answer
27 views

How to make use of ground truth data in image anomaly detection?

If I have an image dataset that consists of "normal", anomalous and ground truth image data, how do I make use of the ground truth data? To my understanding if I train an unsupervised ...
1
vote
1answer
14 views

Train loss value to consider convergence

I am training a fully connected NN with 4 hidden layers for a task of regression of two rational target values, Using MSE loss. My problem is determining whether the training process succeed, that is, ...
0
votes
0answers
25 views

How do they use their dataset with VAEs?

Old Photo Restoration via Deep Latent Space Translation (https://paperswithcode.com/paper/old-photo-restoration-via-deep-latent-space) In the article, it says : "We propose to restore old photos ...
0
votes
0answers
11 views

Normalization and Standardization of color channels for Convolutional Neural Networks

I have created 2D heat maps with 3 color channels. On these heat maps, I will train CNN networks. The range of values in the three colors channels is very different. In the first channel the values ...
2
votes
0answers
20 views

Confusion about the derivative in CTC

I was going through the original CTC paper by Graves et al, I am still not getting how after taking the derivative of equation 14 we get equation 15 as shown below I understand the part that we are ...
0
votes
1answer
15 views

Hyperparameter tuning vs weight tweaking in Cross-Validation: should I consider 2 different validation sets?

Let's say I have 1000 Samples and want to build an ANN. Then I split my dataset into train set (800) and test set (200). After that, I do the following Cross-validate my train set with different ...
1
vote
0answers
28 views

Why do CNNs work well with natural data such as speech, images, and text? [closed]

According to the universal Approximation theorem, we can approximate any given function with two-layer neural networks with a sufficient number of nodes. Then Why do CNNs work well with natural data ...
0
votes
0answers
15 views

Understanding the decision boundary using a tanh activation function in a Neural Network

I was hoping that someone might be able to explain to me a bit why the decision boundary looks the way it does. I believe that this has to do with SGD and this reflects the derivative of the tanh. ...
1
vote
1answer
15 views

ReLU outperforming Softplus

I have noticed that PyTorch models perform significantly better when ReLU is used instead of Softplus with Adam as optimiser. How can it happen to be that a non-differentiable function is easier to ...
1
vote
0answers
10 views

How to define loss function for Discriminator in GANs?

To train the discriminator network in GANs we set the label for the true samples as $1$ and $0$ for fake ones. Then we use binary cross-entropy loss for training. Since we set the label $1$ for true ...
0
votes
0answers
9 views

What should be the ratio of number of classes to number of instances per class?

I am trying to train a CNN model for the classification of 100 different classes. I have about 275 instances for each class and there are about 1000 features. While I trained the model by tuning ...
0
votes
1answer
11 views

Cross-validation for hyperparameter tuning

I've read as many topics regarding hyperparameter tuning as I could, and I developed the following algorithm for hyperparameter tuning & final model building Split the data in train set (80%) &...
1
vote
1answer
33 views

Aging/oldify a dataset of pictures

Old Photo Restoration via Deep Latent Space Translation (https://paperswithcode.com/paper/old-photo-restoration-via-deep-latent-space) My team and I are interested in reproducing their work. However, ...
0
votes
0answers
12 views

Image intensity normalization in preprocessing

Suppose having two images on a given scale, for example it could be the classic [0-255], representing the same thing but with different value intensities, i.e. the first could have a maximum pixel ...
0
votes
0answers
10 views

ML model type for making multi-step time-series predictions

Consider the following problem: making a prediction for 1 month based on 5 years of stock close prices. What would be the best choices in terms of model structure for this problem? I have considered ...
0
votes
0answers
16 views

Faster RCNN for one class in object detection

Let say I have a task to detect the bounding box of one object only. And the only thing I care about is the IoU between prediction and ground truth, no need for real-time. My question: Should I ...
0
votes
0answers
4 views

ML training and test data in making multi-step time-series predictions

Consider the following problem: making a prediction for 1 month based on 5 years of stock close prices. What would be the test data in this problem? The 1 month I would make a prediction for, or ...
0
votes
0answers
12 views

why Alexnet fails even on slight modification to layers [duplicate]

This lines are found in discussion section of Alexnet's Paper. Our results show that a large, deep convolutional neural network is capable of achieving recordbreaking results on a highly challenging ...
6
votes
2answers
205 views

Best way to reduce false positive of binary classification to exactly 0?

I'm working on a task that even a 0.00001 fp rate is not acceptable, because detecting something as a positive when its not will have very bad consequences in this task, so it needs to be exactly 0 ...