All Questions
Tagged with neural-networks tensorflow
381 questions
1
vote
2
answers
34
views
How does a single layer/single unit with Adam optimizer network work?
I'm very new to ML and I'm trying to mess around with Linear Regression. I tested sklearn's LinearRegression model and then wanted to compare the results to a very simple neural network.
I created a ...
- 111
2
votes
1
answer
51
views
How to use differential-entropy as pre-processing?
I am currently working on implementing the model EEG_DMNet. For pre-processing it calls for using differential entropy like
$$
h(X) = -\int_{-\infty}^{\infty} p(x) \log p(x) \, dx
$$
Assuming the Data ...
0
votes
0
answers
75
views
Why the training accuracy stays high but validation accuracy does not change?
I have a binary classification problem. I get ROI mammogram images and then apply a decomposition algorithm and as output I get 5 images which summation of them results in the original image. Now, ...
- 31
1
vote
0
answers
95
views
Dense network can't learn a horizontally shifted log?
I've lately ran into an interesting problem, trying to teach a dense network a seemingly simple monotonous function- to regress a logarithmic function;
When this function was centered around 0 it ...
1
vote
0
answers
20
views
RecSys model performance stalling at 47% AUC and F1-Score. Is the problem due to ratio of users to items in my dataset?
I'm having trouble with making my validation metrics go down for the binary_crossentropy and go up for the F1-score and AUC. I've tried tuning my hyper parameters such as the number of latent features ...
0
votes
1
answer
411
views
Why not use input padding in the first attention block in transformer decoder
I was studying the transformer decoder code below in Keras/Tensorflow. It was not clear how they made making decisions.
In the first attention block below (self.attention_1), why did they use ...
- 1
1
vote
0
answers
32
views
Fluctuating validation accuracy with steady accuracy increase
I have four layers of CNN to predict Javanese script letter data. The training accuracy and loss monotonically increase and decrease respectively. But, my test accuracy starts to fluctuate wildly. I ...
- 11
1
vote
0
answers
252
views
Keras RMSProp what is the alternative to "decay" (no longer available after Keras 2.3)
Background: Hello, I'm creating a GAN with an RMSProp optimizer for both discriminator & generator. The generator model has half the learning rate of the discriminator (1e-4) and half the decay of ...
- 11
1
vote
1
answer
80
views
How to use Activation Functions in Quantized Nerual Networks?
I want to understand how quantized networks can calculate activations like sigmoid and tanh. I stumbled over this question which mentions the implementation of TF-Lite Micro as an example. ...
- 13
1
vote
1
answer
690
views
How to calculate the decay rate given an initial learning rate and final learning rate for schedulers when training neural networks?
I am training a neural network in TensorFlow and I would like to use firstly an exponential decay optimizer scheduler (https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/schedules/...
- 21
0
votes
1
answer
1k
views
Single input - multiple outputs with different loss functions in Keras: how is the gradient computed?
I've implemented a neural network with single input - multiple outputs using Keras API. The general structure of the network is like in this figure:
Because each branch does a different task, I ...
- 87
2
votes
1
answer
548
views
Threshold Tuning before or after parameter tuning?
My goal is to increase the F1 score of Class 1 by 1-2%.
I achieved this by changing the threshold from 0.5 to X using the precision recall curve when the dataset is imbalanced. I did this after I have ...
0
votes
0
answers
164
views
Is it possible to calculate an integral within a layer with tensorflow?
Is it possible to compute an integral within a layer in tensorflow and tensorflow probability? I have a simple MLP with a couple of dense layers and a concat layer.
...
- 325
1
vote
0
answers
232
views
RNN with overlapping timestamp sequences
Maybe a newbie question here, but I’ve not had much experience with sequential models and I’ve not been able to find an example or clear answers to this question online.
All tutorials and resources I ...
1
vote
0
answers
48
views
Data parallelism on multiple GPUs [closed]
I am trying to train a model using data parallelism on multiple GPUs. As I think, in data parallelism, we divide the data into batches, and then batches are deployed parallel. Afterward, the average ...
- 161
1
vote
1
answer
267
views
Use the Same Learning Rate to Train All Models When Doing Experiments For A Deep Learning Paper?
When writing a deep learning paper, I need to train several CNN models and compare their performances. They are from different architectures so different designs.
I'm wondering should I use the same ...
- 111
2
votes
1
answer
466
views
When adding batch norm layer do I need to added to all layers in DNN?
While developing deepfm model network I want to add batch norm layer because model seems to suffer from vanishing gradient. There are embedding layers, 2 layers a in deep model part and one dense ...
- 261
1
vote
0
answers
226
views
Why does the CNN model accuracy vary too much when the dataset is the same?
I have been working on a project where I have a lot of time series data(3000 csv file) from 6 different devices and I am trying to convert those data to an image array so that I can use them in CNN to ...
- 21
1
vote
0
answers
121
views
CycleGAN cycle loss
I was reading the paper of CycleGAN and I was trying to implement it.
However, my models does not converge to any good solution whatsoever, and since I've checked the implementation many times, I ...
- 1,381
2
votes
1
answer
2k
views
How should I train my CNN with a tiny dataset
I'm working on a problem where I aim to classify sections of a track made on the floor using tape, into either left turns, right turns or straight track.
I'm struggling creating a CNN that is not ...
1
vote
1
answer
772
views
Sequential Binary Imbalanced data classification with LSTM
I'm building an LSTM sequential Binary Classification Model, the data is highly imbalanced like say Fraud detection case.
After building an LSTM model on Sequential Vectorised data, I'm getting a very ...
0
votes
0
answers
49
views
Validation performance is really problematic; should I give up on increasing the validation performance of my deep learning model? [duplicate]
I am working on a multiclassification problem using time series data. Three datasets are utilized in this study. My deep neural network performs satisfactorily across two different data sets. However, ...
- 161
1
vote
0
answers
389
views
Classical VAE not learning 2D gaussian mixture distribution using MSE loss
I've been exploring VAE for non-image data. I consider small to medium-sized continuous vector spaces and I want to learn the distribution of a dataset in that space.
As a warm up exercise, I tried ...
- 113
1
vote
1
answer
644
views
How to properly mask MultiHeadAttention for sliding window time series data
I have data in the shape (batch, seq_len, features) that is a time series sliding window. In essence, I'm using the most recent ...
- 185
6
votes
1
answer
3k
views
Neural network gives very different accuracies if repeated on same data, why?
I'm running a neural network to classify audio files into 4 classes. This uses 3300 1min files split roughly evenly across classes. I split this into 80:10:10 train:validation:test sets. This trains ...
- 73
2
votes
0
answers
2k
views
Choosing the 'best' epoch to stop the training of neural network. Top accuracy not improving, but average is
I'm familiar with concepts like early stopping, and detection of plateau and so on.
Tensorflow CNN training has a possibility of saving only best model too, according to model's accuracy metric (for ...
- 121
5
votes
0
answers
1k
views
Using `relu` as activation function for regression with only positive values
I'm building a deep learning model to predict times of arrival. By definition, the time of arrival is always positive. I'm wondering if I can use a relu as the ...
- 288
2
votes
1
answer
2k
views
What's the difference between stacked LSTM and encoder-decoder LSTM
I wanted to learn about encoder-decoder LSTM and after some digging around I get that the first LSTM layer in an encoder-decoder-LSTM outputs its hidden state and then the next LSTM layer uses that ...
- 123
3
votes
1
answer
75
views
Convolutional neural network architecture calculation question
I'm attempting to understand the neural network architecture used in this paper: Visualizing and Understanding Convolutional Networks.
Here's an image of the network achitecture from the paper that is ...
- 33
1
vote
1
answer
282
views
Training MLP by early-stopping without dropout layers
I am training a multi-layer perceptron (MLP) with 4 hidden layers. I got the best hyper-parameters by the following steps using HParams:
Training model by each ...
- 195
1
vote
0
answers
221
views
Getting same prediction for all the classes in mobilenetV2 - Tensorflow
I am using mobile net v2 for multiclass image classification problem,
here is how I am loading the data
...
- 11
0
votes
1
answer
1k
views
Why my training Precision and Recall is higher than Validation Precision and Recall?
I am training a deep learning model for binary image classification using Keras and TensorFlow. My model gave the highest ...
- 119
1
vote
0
answers
21
views
Why my training Precision and Recall is higher than Validation Precision and Recall? [duplicate]
I am training a deep learning model for binary image classification using Keras and TensorFlow. My model gave the highest ...
- 119
4
votes
0
answers
235
views
Fine-tune: ways to determine how many layers to unfreeze
How to determine amount of layers I should unfreeze while fine-tuning deep learning model? Is there any sets of rules or I should just experiment?
- 41
6
votes
3
answers
3k
views
Impose a condition on neural network
I am building a neural network model with TensorFlow and Keras in python. My model is performing well on unseen data in the way I desire and everything is fine. but the problem that I don't have any ...
- 113
1
vote
0
answers
337
views
How to apply Keras Conv1D over 3D dimensional input?
Context: I'm predicting whether a machine will break down within 1 hour, and I have sensors located at 4 different parts of the machine, which give me historical readings of different metrics. ...
- 123
1
vote
0
answers
868
views
How to deal with zeros in neural network? [closed]
I am working on a Keras and TensorFlow in R and I try to make good predictions for a regression problem (not classification).
In my dataset there are several features and my target variable has a lot ...
- 143
0
votes
1
answer
478
views
Is it possible in deep learning to train on a subset of training set in order to find the best hyper-parameters?
In classic machine learning, it is not uncommon to do a search for hyper-parameters by training different configurations on a small subset of training set. Usually, for each set of hyper-parameters, a ...
- 101
17
votes
6
answers
3k
views
Can I enforce monotonically increasing neural net outputs (min, mean, max)?
Hi I'm using a DL model (TensorFlow) to predict daily minimum, mean, and maximum values of a target dataset. I was thinking that the model would have 3 outputs for each day, (min, mean, max).
Is there ...
- 271
6
votes
1
answer
3k
views
How to train a neural network to minimize two loss functions?
For TF/Keras (or in general), what is the best way to define a multidimensional y target? Should this even be done?
The problem:
Any sample x tries to predict several "values of interest". ...
- 163
1
vote
0
answers
7
views
Is it possible to train CNN on 1 column of csv data? [duplicate]
I want to train CNN model on 1 column data, the data was taken from EMG Sensor the shape of data represented in one column such that:
these data represent the EMG signals,
column 0 is signals value - ...
1
vote
1
answer
41
views
can a trained tensorflow neural network make predictions on data without annotations? [closed]
I have effectively trained a sequential neural network using tensorflow on an annotated NER corpus. I'm trying to understand how I can feed unannotated data into the model so I can get predicted named ...
- 11
1
vote
1
answer
395
views
LSTM: How Do I Predict A Single Label Multiple Steps Ahead?
I'm building an LSTM neural network using the Tensorflow tutorial below.
https://www.tensorflow.org/tutorials/structured_data/time_series#single-shot_models
It shows you how to build single-step ...
- 11
0
votes
0
answers
49
views
Query regarding Deep learning model performance reporting
I am working on Human activity recognition via smart device sensors data by using deep learning. However, I am confused to report the results of my deep learning architecture. Therefore, I would like ...
- 63
0
votes
2
answers
178
views
Which accuracy is called the accuracy of a deep learning model?
I have developed a deep learning model, to predict whether an image is affected by a certain disease or not. Accuracies of 99.8%, 88.8%, and 89% have been achieved on the training set, testing set, ...
1
vote
1
answer
293
views
Hand Keypoint Detection Model Not Converging
I'm currently trying to train a custom model with tensorflow to detect 17 landmarks/keypoints on each of 2 hands shown in an image (fingertips, first knuckles, bottom knuckles, wrist, and palm), for ...
- 111
1
vote
0
answers
256
views
Dense layer with more neurons than on input
What is the purpose of having a dense layer with more neurons on the output than it received on input?
Let's imagine we have a neural network in which the last layer has the size of 1. Does it make ...
- 11
0
votes
1
answer
504
views
How does tf.keras.MultiHeadAttention layer handle positional encoding?
In Attention Is All You Need paper, positional encodings are added to the input embeddings in order to consider the order of the sequence. How does tf.keras.MultiHeadAttention handle positional ...
1
vote
0
answers
347
views
Visualization of fully-connected neural networks
I'm trying to visualize a neural network built by keras with the following structure where all three Dense layers include a bias ...
- 91
1
vote
1
answer
56
views
How Do You Know If A Problem Set Can Be Trained?
As the title suggests, how do you know if there exists a machine learning solution to a problem set? Earlier today I was working on building a neural network to predict whether stock prices will going ...
- 13