Questions tagged [tensorflow]

A Python library for deep learning developed by Google. Use this tag for any on-topic question that (a) involves tensorflow either as a critical part of the question or expected answer, & (b) is not just about how to use tensorflow.

Filter by
Sorted by
Tagged with
46
votes
3answers
68k views

Should I use a categorical cross-entropy or binary cross-entropy loss for binary predictions?

First of all, I realized if I need to perform binary predictions, I have to create at least two classes through performing a one-hot-encoding. Is this correct? However, is binary cross-entropy only ...
34
votes
6answers
27k views

Understanding LSTM units vs. cells

I have been studying LSTMs for a while. I understand at a high level how everything works. However, going to implement them using Tensorflow I've noticed that BasicLSTMCell requires a number of units (...
33
votes
3answers
13k views

Building an autoencoder in Tensorflow to surpass PCA

Hinton and Salakhutdinov in Reducing the Dimensionality of Data with Neural Networks, Science 2006 proposed a non-linear PCA through the use of a deep autoencoder. I have tried to build and train a ...
31
votes
1answer
12k views

Step-by-step example of reverse-mode automatic differentiation

Not sure if this question belongs here, but it's closely related to gradient methods in optimization, which seems to be on-topic here. Anyway, feel free to migrate if you think some other community ...
25
votes
1answer
26k views

Loss function for autoencoders

I am experimenting a bit autoencoders, and with tensorflow I created a model that tries to reconstruct the MNIST dataset. My network is very simple: X, e1, e2, d1, Y, where e1 and e2 are encoding ...
3
votes
1answer
180 views

Training N classifiers for N labels vs one classifier with N labels

I have a classification problem which is multi-label with N labels. I would like to know which method would be the better choice? Training N classifiers (1 for each label) or a single classifier which ...
2
votes
1answer
33 views

Numerical computation of cross entropy in practice

The equation for cross-entropy is: $H(p,q)=-\sum_x{p(x)\log{q(x)}}$ When working with a binary classification problem, the ground truth is often provided to us as binary (i.e. 1's and 0's). If I ...
39
votes
1answer
40k views

CNN architectures for regression?

I've been working on a regression problem where the input is an image, and the label is a continuous value between 80 and 350. The images are of some chemicals after a reaction takes place. The color ...
32
votes
2answers
14k views

how to weight KLD loss vs reconstruction loss in variational auto-encoder

in nearly all code examples I've seen of a VAE, the loss functions are defined as follows (this is tensorflow code, but I've seen similar for theano, torch etc. It's also for a convnet, but that's ...
20
votes
3answers
26k views

Difference between samples, time steps and features in neural network

I am going through the following blog on LSTM neural network: http://machinelearningmastery.com/understanding-stateful-lstm-recurrent-neural-networks-python-keras/ The author reshapes the input ...
20
votes
5answers
27k views

Deep learning : How do I know which variables are important?

In terms of neural network lingo (y = Weight * x + bias) how would I know which variables are more important than others? I have a neural network with 10 inputs, 1 hidden layer with 20 nodes, and 1 ...
14
votes
5answers
11k views

Does the cross-entropy cost make sense in the context of regression?

Does the cross-entropy cost make sense in the context of regression (as opposed to classification)? If so, could you give a toy example through TensorFlow? If not, why not? I was reading about cross-...
10
votes
3answers
7k views

Loss function autoencoder vs variational-autoencoder or MSE-loss vs binary-cross-entropy-loss

When having real valued entries (e.g. floats between 0 and 1 as normalized representation for greyscale values from 0 to 256) in our label vector, I always thought that we use MSE(R2-loss) if we want ...
2
votes
3answers
7k views

How to increase accuracy of All-CNN C on CIFAR-10 test set [closed]

I am trying to implement the paper Striving for Simplicity specifically the model All-CNN C on CIFAR-10 without data augmentation. This model is said to be able to reach close to 91% accuracy on test ...
14
votes
2answers
29k views

Keras: why does loss decrease while val_loss increase?

I setup a grid search for a bunch of params. I am trying to find the best parameters for a Keras neural net that does binary classification. The output is either a 1 or a 0. There are about 200 ...
6
votes
1answer
4k views

Difference Between Rho and Decay Arguments in Keras RMSprop

I am working to tune a RNN for the purposes of predictive analytics on time series data. I am testing different optimizers and am currently working with RMSprop. I have reviewed Hinton's lecture ...
4
votes
1answer
625 views

Loss function (and encoding?) for angles

I'm training a network to predict the angle of arrival of a signal. Labels are single values in the [-180, 180) interval. I'm seeing a discontinuity in predictions around ±180 degrees, which makes ...
1
vote
0answers
321 views

Xavier initialisation of weights for 3D convolutions

I am using 3D convolutions in tensorflow. I want to initialise the weights using Xavier initialisation. Tensorflow has an implementation of Xavier initialisation for 2D convolutions, from this ...
6
votes
1answer
2k views

Why 'e' in softmax?

I am doing an introduction to ML with tensorflow and I came across softmax activation function. Why is in the softmax formula e? Why not 2? 3? 7? $$ \text{softmax}(x)_i = \frac{\exp(x_i)}{\sum_j \exp(...
2
votes
0answers
1k views

Discrepancy between categorical cross entropy and classification accuracy

I have a convolution neural network with random weights initialized and Trained to perform binary classification. I have 2000 images as training data and 2000 validation data. The problem I am trying ...
2
votes
1answer
1k views

Back propagation in seq2seq models

I've implemented a seq2seq model for character-based text mirroring as a part of Udacity's Deep Learning class (here's the code). My model is very basic because it's a single LSTM as both encoder and ...
2
votes
1answer
7k views

OCR model with TensorFlow

I'm just getting started in Machine Learning and I'd like to implement a simple OCR. I've played with the MNIST dataset, but it was classification over a finite number of classes, and one character ...
1
vote
1answer
1k views

Need advice for creating a good custom loss function for a neural network

I'm trying to solve a regression problem using a neural network. In my problem domain, an underestimation is a lot worse than an overestimation, so I thought I'd create a custom loss function for my ...
5
votes
4answers
6k views

Difference between strided and non-strided convolution

conv = conv_2d (strides=) I want to know in what sense a non-strided convolution differs from a strided convolution. I know how convolutions with strides work but ...
1
vote
1answer
1k views

How to train a CNN with non-squared data?

I'm having serious problems with this task. It already has weeks that I'm trying to achiev a way to train a non-squared (256x16, for example) data. How could I do that? I'm trying to apply conv with ...
1
vote
0answers
95 views

DCGAN: How to nearest neighbor on the training set in sense of compressed sensing?

I want to find the nearest neighbor of the output compared to the training set based on this framework: https://github.com/carpedm20/DCGAN-tensorflow (I asked already generally in this post: ...
1
vote
1answer
189 views

Reproducible numbers in Keras/TensorFlow

Every time I run a Keras/TensorFlow code gives different results. Can someone suggest how to get reproducible numbers?
0
votes
0answers
164 views

NN Accuracy and Understanding tensorflow-graphs

If I test accuracy using the below code while training, then it works well. If however, I use a separate function then it outputs the wrong value (about 20 times too small). There must be something ...