Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Questions tagged [keras]

Open source high-level neural network library for Python and R. Is capable of using TensorFlow or Theano as backend.

Filter by
Sorted by
Tagged with
180
votes
4answers
261k views

What is batch size in neural network?

I'm using Python Keras package for neural network. This is the link. Is batch_size equals to number of test samples? From ...
70
votes
3answers
47k views

How does Keras 'Embedding' layer work?

Need to understand the working of 'Embedding' layer in Keras library. I execute the following code in Python ...
65
votes
6answers
70k views

What loss function for multi-class, multi-label classification tasks in neural networks?

I'm training a neural network to classify a set of objects into n-classes. Each object can belong to multiple classes at the same time (multi-class, multi-label). I read that for multi-class problems ...
21
votes
5answers
9k views

How do I make my neural network better at predicting sine waves?

Here, have a look: You can see exactly where the training data ends. Training data goes from $-1$ to $1$. I used Keras and a 1-100-100-2 dense network with tanh activation. I calculate the result ...
20
votes
3answers
25k views

Understanding input_shape parameter in LSTM with Keras

I'm trying to use the example described in the Keras documentation named "Stacked LSTM for sequence classification" (see code below) and can't figure out the ...
14
votes
3answers
5k views

Is it possible to give variable sized images as input to a convolutional neural network?

Can we give images with variable size as input to a convolutional neural network for object detection? If possible, how can we do that? But if we try to crop the image, we will be loosing some ...
14
votes
3answers
17k views

What is the difference between Conv1D and Conv2D?

I was going through the keras convolution docs and I have found two types of convultuion Conv1D and Conv2D. I did some web search and this is what I understands about Conv1D and Conv2D; Conv1D is used ...
13
votes
1answer
1k views

Why can't a single ReLU learn a ReLU?

As a follow-up to My neural network can't even learn Euclidean distance I simplified even more and tried to train a single ReLU (with random weight) to a single ReLU. This is the simplest network ...
11
votes
3answers
16k views

CIFAR-10 Can't get above 60% Accuracy, Keras with Tensorflow backend [closed]

Training after 15 epochs on the CIFAR-10 dataset seems to make the validation loss no longer decrease, sticking around 1.4 (with 60% validation accuracy). I've shuffled the training set, divided it by ...
11
votes
2answers
2k views

Difference between a single unit LSTM and 3-unit LSTM neural network

The LSTM in the following Keras code input_t = Input((4, 1)) output_t = LSTM(1)(input_t) model = Model(inputs=input_t, outputs=output_t) print(model.summary()) ...
10
votes
3answers
8k views

How the embedding layer is trained in Keras Embedding layer

How is the embedding layer trained in Keras Embedding layer? (say using tensorflow backend, meaning is it similar to word2vec, glove or fasttext) Assume we do not use a pretrained embedding.
10
votes
2answers
14k views

How to set mini-batch size in SGD in keras

I am new to Keras and need your help. I am training a neural net in Keras and my loss function is Squared Difference b/w net's output and target value. I want to optimize this using Gradient Descent....
9
votes
2answers
17k views

Simple Linear Regression in Keras

After looking at This question: Trying to Emulate Linear Regression using Keras, I've tried to roll my own example, just for study purposes and to develop my intuition. I downloaded a simple dataset ...
9
votes
1answer
628 views

My neural network can't even learn Euclidean distance

So I'm trying to teach myself neural networks (for regression applications, not classifying pictures of cats). My first experiments were training a network to implement an FIR filter and a Discrete ...
8
votes
2answers
4k views

Is there a way to incorporate new data into an already trained neural network without retraining on all my data in Keras?

I have already trained a neural network on my data. In the future, I will receive some more data. How can I incorporate this data into my model without rebuilding it from scratch?
8
votes
3answers
5k views

Why does the loss/accuracy fluctuate during the training? (Keras, LSTM)

I use LSTM network in Keras. During the training, the loss fluctuates a lot, and I do not understand why that would happen. Here is the NN I was using initially: And here are the loss&accuracy ...
7
votes
1answer
24k views

Why does keras binary_crossentropy loss function return wrong values? [closed]

Binary cross entropy for multi-label classification can be defined by the following loss function: $$-\frac{1}{N}\sum_{i=1}^N [y_i \log(\hat{y}_i)+(1-y_i) \log(1-\hat{y}_i)]$$ Why does keras ...
7
votes
1answer
5k views

Training LSTM with and without resetting states

I'm quite new to deep learning and Keras and I want to know what is the difference between these two training methods of an LSTM RNN. ...
7
votes
2answers
844 views

Why LSTM performs worse in information latching than vanilla recurrent neuron network

I would like to understand better why LSTM can remember information for a longer time period than vanilla/simple recurrent neural network (SRNN) by redoing an experiment from the paper Learning Long-...
7
votes
2answers
8k views

Epochs in keras meaning? [closed]

What does this mean? Epoch 1/300 7200/7200 [==============================] - 0s - loss: 3.3616 - acc: 0.3707 I built a neural network in keras and this is ...
7
votes
1answer
3k views

How to prepare data for input to a sparse categorical cross entropy multiclassification model [closed]

So I have a set of Tweets with a few columns such as Date and the Tweet itself and a few more but I want to use 2 columns to build my model(Sentiment & Stock Price) Sentiment analysis is performed ...
6
votes
2answers
5k views

Difference between kernel, bias, and activity regulizers in Keras

I've read this post, but I wanted more clarification for a broader question. In Keras, there are now three types of regularizers for a layer: kernel_regularizer, <...
6
votes
2answers
6k views

Overfitting in neural network

I am a newbie to neural network. i am using TensorFlow + Keras to model my neural network for classification of 12 logos. The model has 5 convolution layers. I have trained a neural network model and ...
6
votes
1answer
305 views

Can small SGD batch size lead to faster overfitting?

I have feedforward neural net, trained on cca 34k samples and tested on 8k samples. There is 139 features in dataset. The ANN does classification between two labels, 0 and 1, so I am using sigmoid ...
5
votes
2answers
222 views

Deep Learning: Wild differences after model is retrained on the same data, what to do?

I am using keras to train a 5 layer regression model to predict 1000 different thermometers. I train a model and then ask it to predict what the reading will be ...
5
votes
2answers
905 views

feature extraction: freezing convolutional base vs. training on extracted features

[Note: To clarify, this question is concerned about the theory and the codes are only used to better explain the issue. This is not in any way a programming question.] In section 5.3 of "Deep ...
5
votes
1answer
2k views

What is the meaning of fuzz factor?

I came across this documentation in keras for the list of backend functions One of which was keras.backend.epsilon() The documentation says that it returns the ...
5
votes
1answer
2k views

LSTM NN produces “shifted” forecast (low quality result)

I am trying to see the power of recurrent neural calculations. I give the NN just one feature, a timeseries datum one step in the past, and predict a current datum. The timeseries is however double-...
5
votes
1answer
5k views

Understanding how to batch and feed data into a stateful LSTM

Let me use daily price prediction of Bitcoin as a simple example (I am not actually working with Bitcoin but its temporal nature fits well to explain my question). Say I had a data set consisting of ...
5
votes
1answer
1k views

Keras difference between GRU and GRUCell

In Keras documentation at this page, https://keras.io/layers/recurrent/, I wonder what's the difference between GRU and GRUCell, which one I should use if I'd like to create a GRU recurrent network. ...
4
votes
2answers
15k views

How to use pre trained word2vec model?

Where can I find a reliable word2vec model trained on some English articles? I need a word2vec black box, where I, for example, ...
4
votes
1answer
4k views

Batch normalisation at the end of each layer and not the input?

I am currently studying the paper of network implementation RCNN. The core module inside RCNN is the Recurrent Convolutional Layer (RCL), whose state evolves over discrete time steps. The ...
4
votes
1answer
3k views

Difference Between Rho and Decay Arguments in Keras RMSprop

I am working to tune a RNN for the purposes of predictive analytics on time series data. I am testing different optimizers and am currently working with RMSprop. I have reviewed Hinton's lecture ...
4
votes
1answer
2k views

How does Keras generate an LSTM layer. What's the dimensionality? [closed]

In Keras I can define the input shape of an LSTM (and GRU) layers by defining the number of training data sets inside my batch (batch_size), the number of time steps and the number of features. So I ...
4
votes
1answer
900 views

Bad performance with ReLU activation function on MNIST data set

I'm quite new to neural networks and currently I'm trying to train a non convolutional neural network on the MNIST data set. I'm observing some behaviour I don't quite understand. This is the code ...
4
votes
2answers
2k views

How to determine what type of layers do I need for my Deep learning model?

Suppose that I have want to make a model that does something. Now when I search about the topic in Google or YouTube, I find many related tutorials and it seems like some clever programmer had already ...
4
votes
1answer
412 views

More Loss in Training than Testing using multi-layer LSTM Neural Networkin Keras/TF

Unsure why I'm consistently seeing a higher training loss than test loss in my model: ...
4
votes
1answer
722 views

Neural Networks - Performance VS Amount of Data

This is one of the slides from Andrew Ng course on deep learning. Actually I took it from Jason Brownlee website that seems to second the idea presented on the picture. However, my limited experience ...
4
votes
3answers
4k views

What is difference between keras embedding layer and word2vec?

In other words, is there a paper that describes the method of keras embedding layer? Is there a comparison between these methods (and other methods like Glove etc.)?
4
votes
1answer
791 views

Neural network regression with confidence interval implemented with Keras

When using neural network for classification problem, and using softmax as last layer for last layer. Typically, we have a prediction and a confidence level. However, is there such confidence ...
4
votes
1answer
11k views

How to get continuous output with Convolutional network? (Keras) [closed]

I'm new in using convolutional neural networks with keras. I can train a CNN for classify somethings and in other words for discrete output, but I can't find an example for getting continuous output (...
4
votes
1answer
320 views

Predicting the observations in a POMDP with a recurrent neural network

I use neural networks for online sequence prediction. The performance of LSTM in this case, however, is not nearly as good as I expected. Maybe someone can help me understand where the problem lies. ...
4
votes
0answers
60 views

Time series predictions look suspiciously good [closed]

I am working on a time series forecasting problem. For this, I am training a recurrent neural network in Keras (mostly following the guidelines from this blog post by Jason Brownlee). My problem ...
4
votes
0answers
390 views

traditional state-space models and LSTMs

I am trying to understand the nature of LSTMs in relation to intuitions from traditional state-space models (e.g., Kalman filtering). The code below aims to simulate a simple univariate linear state-...
4
votes
1answer
777 views

Trouble training LSTM for sequence to sequence learning of sensor time series

I'm experimenting with using RNNs/LSTMs in place of a Kalman Filter (KF) for sensor fusion. I'm struggling to make much progress, and would appreciate some feedback/advice. I have several multi-...
4
votes
0answers
449 views

Advantage of fit_generator() in keras [closed]

I was wondering if the fit_generator() in keras has any advantage in respect to memory usage over using the usual fit() method ...
3
votes
1answer
323 views

Does it make sense to use an Early Stopping Metric like “mae” instaed of “val_loss” for regression problems?

I am performing a regression on a Dataset and try to replace a mathematical Model with a Neural Network. To avoid overfitting I decided to use the Early Stopping Callback Function of Keras. So far I ...
3
votes
1answer
2k views

Number of trainable parameters in Convolution models (Keras)

I am using keras to implement a cnn model. ...
3
votes
1answer
5k views

LSTM clarification on output

I (think), I understand how LSTMs are roughly working. I need some clarifications though. Since there can be an output at each time step, the "output dimension" (in the sense of many to one or many ...
3
votes
1answer
163 views

Loss function (and encoding?) for angles

I'm training a network to predict the angle of arrival of a signal. Labels are single values in the [-180, 180) interval. I'm seeing a discontinuity in predictions around ±180 degrees, which makes ...