Questions tagged [theano]

Python library for parallel GPU-based computations with multidimensional arrays. Often used to implement deep neural networks.

Filter by
Sorted by
Tagged with
0
votes
0answers
20 views

Wrong weights learned when training RBM

I'm training my RBM network and on epoch #4 I have such a filters representation (my weights matrix) But on the next iteration (fifth epoch) something went wrong and my filters became like this What ...
0
votes
1answer
56 views

What is the difference between a deep learning library that is built for research and for production?

I always see things like "a library for research but not so much for production" What does that mean? What should i choose? Can i built a an advanced model (ex: to compete in image-net) using anyone ...
5
votes
2answers
221 views

Deep Learning: Wild differences after model is retrained on the same data, what to do?

I am using keras to train a 5 layer regression model to predict 1000 different thermometers. I train a model and then ask it to predict what the reading will be ...
1
vote
2answers
873 views

How to extract the features making up the hidden layers in Autoencoders

Apologies if this question has been asked before but I haven't come across any so far. I have been experimenting with Autoencoders using Keras and Theano as my back end based on the tutorial from ...
2
votes
0answers
281 views

Different sized inputs for batch training in fully CNNs

The idea of transfer learning is to use already trained networks for another purposes to the one it was initially trained for. Using fully convolutional networks, the activation maps can be used to ...
1
vote
0answers
305 views

LSTM is converging to the same prediction at every timestep [duplicate]

I see that this is essentially a duplicate of LSTM Classifying all the words as the same class, but I thought that after roughly two years, someone might have insight into this problem. I'm trying ...
1
vote
1answer
2k views

Convolutional neural networks - What is done first? Padding or convolving?

Convolutional neural networks - What is done first? Padding or convolving? Suppose the code below : ...
0
votes
0answers
202 views

Theano K Means with Shared Variables and Scan

I have a pet project to reproduce some common clustering in theano in order to improve my understanding for future projects. I was wondering if anyone has ever used nested theano scans on shared ...
3
votes
1answer
782 views

LSTM Weight Matrix Interpretation

Consider the following code in Keras for building a LSTM model. ...
13
votes
2answers
5k views

Neural Nets: One-hot variable overwhelming continuous?

I have raw data that has about 20 columns (20 features). Ten of them are continuous data and 10 of them are categorical. Some of the categorical data can have like 50 different values (U.S. States). ...
0
votes
0answers
238 views

Implement Tree-structure computation graph in Theano

I would like to implement TreeLSTM in Theano run on tree-structure dataset (like Sentiment Treebank). However, I have trouble implement the tree structure. I have done a simple example of the tree-...
18
votes
5answers
20k views

Deep learning : How do I know which variables are important?

In terms of neural network lingo (y = Weight * x + bias) how would I know which variables are more important than others? I have a neural network with 10 inputs, 1 hidden layer with 20 nodes, and 1 ...
0
votes
1answer
417 views

Keras : Should I care about the LAST epoch val_acc or only the HIGHEST epoch val_acc?

I am running a grid search for a binary classifier. Each model will receive a list of parameters and then that model will be run 20 times (20 epochs.) Many times the first epoch will have a high ...
11
votes
2answers
19k views

Keras: why does loss decrease while val_loss increase?

I setup a grid search for a bunch of params. I am trying to find the best parameters for a Keras neural net that does binary classification. The output is either a 1 or a 0. There are about 200 ...
1
vote
1answer
8k views

Keras: acc and val_acc are constant over 300 epochs, is this normal?

I am trying to understand a relationship between some x-cols and a y-col. There are about 25 features, some of which are categorical type. After a one-hot transformation on the categorical x-cols, the ...
1
vote
1answer
4k views

Keras Numpy Error: Why does accuracy go to zero?

I am trying to follow this tutorial. Everything works when I copy and paste the data, but when I try to replace the dataset with a random numpy matrix, my accuracy goes to zero. It shouldn't go to ...
2
votes
1answer
79 views

Neural networks, general inquiry - how mini-batching fits into the big picture?

I am graduate student in math, it is my first look at the Deep learning. One of the things that strike me when looking at this material is what seems to be a confusion between implementation details ...
3
votes
0answers
1k views

Recurrent Neural Network on Panel Data

There are 2 parts to this question. Suppose we are looking at sales $S$ of a product across $> 1000$ stores where a it sells. For each of these $1000$ stores we have 24 months recorded data. We ...
4
votes
1answer
611 views

Keras Theano conv/deconv autoencoder dimension mis-match

I'm trying to build an autoencoder in keras based on examples in this blog post but with different layers. The model compiles but throws an error (Theano) ...
1
vote
2answers
3k views

Sequence classification via Neural Networks

What exact kind of architecture of neural networks do I need for a sequence binary/multiclass classification? The sequences can be of different length and are to be discriminated by a certain ...
1
vote
0answers
85 views

Neural network hyperparameters don't affect performance [duplicate]

I am trying to perform multi-label classification using neural network, with 3 hidden ReLU layers and a softmax output layer. I'm also using mini-batch SGD as the optimization algorithm and negative ...
2
votes
1answer
956 views

How can I re-code this hierarchical model in PyMC 3?

I wish to model data from an experiment using a hierarchical Bayesian logistic regression. The experiment involved many subjects, and many trials collected from each subject. The DV is the outcome of ...
2
votes
0answers
444 views

Autoencoder not able to generate same input and output value

I am doing some work with Theano based auto-encoder, giving input as samples from mixture of Gaussians, one hidden layer. I expected output to be same as input, but I am not achieving it. I have been ...
3
votes
2answers
312 views

How are convolutional layers connected in Theano?

How are feature maps connected between two layers in Theano/Caffe/TensorFlow? For instance, if we have 32 feature maps in Conv Layer 1, and 64 feature maps in Conv Layer 2, with 64 kernels, how does ...
1
vote
1answer
928 views

Deep Neural Network tuning hyperparameters

I want to find the most efficient Feed forward Deep Neural Network architecture for my problem (binary classification). Study roadmap: Create network contains only one hidden layer, Tune ...
4
votes
2answers
1k views

Why do Deep Learning libraries force the cost function to output a scalar?

Let's say we have a neural net with: 5 input neurons some arbitrary amount of hidden layers 3 output neurons Let's say we're using minibatches of size 32. So, if we input a 5x32 matrix into the ...
1
vote
2answers
2k views

Stopping Exploding Gradients in Keras

I have a LSTM (Long-Short Term Memory) Neural Network that has this structure: ...
0
votes
1answer
3k views

Stacked LSTM for sequence classification - Keras - Error [closed]

Hi I am trying to use a stacked LSTM architecture in keras, similar to what is shown here https://keras.io/getting-started/sequential-model-guide/. My problem is formulated as a binary time series ...
1
vote
0answers
202 views

Logistic Tensor Regression

Say there are users and they view articles and click (or not click) on articles. I represent the $i$-th user as $x_i$, a $D \times1 $ vector and $j$-th article as $z_j$, a $C \times 1$vector. The ...
1
vote
1answer
731 views

Implementing backpropagation in Theano

I am new to the machine learning field, so I am not sure if I am asking a dumb question. I have been playing around with Theano for a while and read a lot of code examples and it looks like every time ...
4
votes
1answer
202 views

If we have auto differentiate tool, do we also need EM algorithm?

I my opinion, EM algorithm is used to estimate the parameters of some complex log likelihood function. Because sometimes, it's hard to get the derivative, we can use EM algorithm. But if we have some ...
1
vote
1answer
102 views

Commutative function mapping sequence of vectors to vector

There is sequence of vectors $[v_1, v_2, v_3]$ passed as input to deep learning regression model $F$. As observed, the order of this sequence is irrelevant, so $F([v_1, v_2, v_3]) = F([v_2, v_1, v_3])...
1
vote
1answer
68 views

Two ways of optimize the same function

i m reading actually this tutorial about deepLearning and in particurlar about Logistic Regression. http://deeplearning.net/tutorial/logreg.html#logreg I don't get why it first says to optimize ...
2
votes
0answers
69 views

Perceptron trained on time series always predicting the same answer [duplicate]

Using the model from theano's tutorial, I'm training a 3-layers perceptron with log returns over a very large dataset (~55,000 points). The output's layer contains two neurons, one for each of the ...
1
vote
0answers
880 views

Simple NN with Lasagne and NoLearn

I wanted to test nolearn and Lasagne with a simple neural network and train the XOR problem, but I run into odd problems. The code below fails with the error ...
1
vote
0answers
96 views

How to generate n theano.shared variables for Gaussian mixture regression?

I am trying to program a Gaussian mixture regression using python, and the theano package. Suppose I have n clusters, then I need to generate n regression weight matrices for each cluster (also, n ...
4
votes
1answer
934 views

What is the purpose of the scaling factor used in dropout?

I have a question related to the dropout function in the LSTM tutorial: http://deeplearning.net/tutorial/code/lstm.py ...
0
votes
1answer
804 views

Initializing starting weights in Neural Nets with bounds?

I'm implementing a deep denoising autoencoder using Theano in Python. I know that you are supposed to initialise starting weights using random numbers and that's what Theano authors do in their ...
2
votes
1answer
73 views

Get confidence of net in theanets [closed]

In theanets, how can I get the net's "confidence" that it is correct? Specifically, I have two output neurons, for two classes, how can I get their values when I predict on an input?
3
votes
3answers
1k views

Recursive neural network implementation in Theano [closed]

Is there any available recursive neural network implementation in Theano? Theano's deeplearning.net tutorial does not present any recursive neural networks. Most Theano code I've found is CNN, LSTM, ...
1
vote
1answer
144 views

Code for minimalistic neural net with one hidden layer [closed]

I am new to theano and learning. Am I doing regularisation correctly? Is this term the correct L2 norm? 0.01 * (wh ** 2).sum() + 0.01 * (wy ** 2).sum() Any ...
3
votes
0answers
722 views

Nesterov vs. momentum gradient descent

I implemented these two methods in a deep learning project where I am using theano. I understand the mathematical difference between these two methods, and my conceptual understanding is that ...
4
votes
2answers
4k views

Robust softmax solutions for Theano?

I am implementing multilayer perceptrons with the softmax activation function over Theano. In some extreme cases I am running into problems with too high/low values in the softmax function that ...
3
votes
1answer
451 views

Stackable LSTM layer trained with arbitrary BPTT time steps

Anyone knows how to make a LSTM layer that is able to be trained with arbitrary BPTT time steps and easy to be stacked together? I am now implementing a basic version of LSTM layer. My scan function ...
7
votes
1answer
4k views

How do I implement a deep autoencoder?

I'm trying to replicate results of this paper using Theano. The problem at the moment is, all Theano-related tutorials are only for MNIST classifiers, which isn't much use in unsupervised image ...