Questions tagged [theano]

Python library for parallel GPU-based computations with multidimensional arrays. Often used to implement deep neural networks.

Filter by
Sorted by
Tagged with
1 vote
1 answer
49 views

What materials are a starting point for developing deep learning architectures?

So some background on my existing knowledge. I have my masters in statistics where I spent a good amount of time understanding how machine learning algorithms work, but I was always allowed to use ...
user avatar
1 vote
1 answer
60 views

What is the difference between a deep learning library that is built for research and for production?

I always see things like "a library for research but not so much for production" What does that mean? What should i choose? Can i built a an advanced model (ex: to compete in image-net) using anyone ...
user avatar
  • 1,250
6 votes
2 answers
771 views

Deep Learning: Wild differences after model is retrained on the same data, what to do?

I am using keras to train a 5 layer regression model to predict 1000 different thermometers. I train a model and then ask it to predict what the reading will be ...
user avatar
1 vote
2 answers
1k views

How to extract the features making up the hidden layers in Autoencoders

Apologies if this question has been asked before but I haven't come across any so far. I have been experimenting with Autoencoders using Keras and Theano as my back end based on the tutorial from ...
user avatar
  • 11
2 votes
0 answers
299 views

Different sized inputs for batch training in fully CNNs

The idea of transfer learning is to use already trained networks for another purposes to the one it was initially trained for. Using fully convolutional networks, the activation maps can be used to ...
user avatar
2 votes
1 answer
3k views

Convolutional neural networks - What is done first? Padding or convolving?

Convolutional neural networks - What is done first? Padding or convolving? Suppose the code below : ...
user avatar
0 votes
0 answers
244 views

Theano K Means with Shared Variables and Scan

I have a pet project to reproduce some common clustering in theano in order to improve my understanding for future projects. I was wondering if anyone has ever used nested theano scans on shared ...
user avatar
  • 1
2 votes
0 answers
1k views

LSTM Weight Matrix Interpretation

Consider the following code in Keras for building a LSTM model. ...
user avatar
14 votes
2 answers
8k views

Neural Nets: One-hot variable overwhelming continuous?

I have raw data that has about 20 columns (20 features). Ten of them are continuous data and 10 of them are categorical. Some of the categorical data can have like 50 different values (U.S. States). ...
user avatar
0 votes
0 answers
269 views

Implement Tree-structure computation graph in Theano

I would like to implement TreeLSTM in Theano run on tree-structure dataset (like Sentiment Treebank). However, I have trouble implement the tree structure. I have done a simple example of the tree-...
user avatar
27 votes
6 answers
37k views

Deep learning : How do I know which variables are important?

In terms of neural network lingo (y = Weight * x + bias) how would I know which variables are more important than others? I have a neural network with 10 inputs, 1 hidden layer with 20 nodes, and 1 ...
user avatar
2 votes
1 answer
1k views

Keras : Should I care about the LAST epoch val_acc or only the HIGHEST epoch val_acc?

I am running a grid search for a binary classifier. Each model will receive a list of parameters and then that model will be run 20 times (20 epochs.) Many times the first epoch will have a high ...
user avatar
22 votes
2 answers
53k views

Keras: why does loss decrease while val_loss increase?

I setup a grid search for a bunch of params. I am trying to find the best parameters for a Keras neural net that does binary classification. The output is either a 1 or a 0. There are about 200 ...
user avatar
2 votes
1 answer
15k views

Keras: acc and val_acc are constant over 300 epochs, is this normal?

I am trying to understand a relationship between some x-cols and a y-col. There are about 25 features, some of which are categorical type. After a one-hot transformation on the categorical x-cols, the ...
user avatar
4 votes
1 answer
8k views

Keras Numpy Error: Why does accuracy go to zero?

I am trying to follow this tutorial. Everything works when I copy and paste the data, but when I try to replace the dataset with a random numpy matrix, my accuracy goes to zero. It shouldn't go to ...
user avatar
2 votes
1 answer
116 views

Neural networks, general inquiry - how mini-batching fits into the big picture?

I am graduate student in math, it is my first look at the Deep learning. One of the things that strike me when looking at this material is what seems to be a confusion between implementation details ...
user avatar
  • 680
3 votes
0 answers
1k views

Recurrent Neural Network on Panel Data

There are 2 parts to this question. Suppose we are looking at sales $S$ of a product across $> 1000$ stores where a it sells. For each of these $1000$ stores we have 24 months recorded data. We ...
user avatar
4 votes
1 answer
641 views

Keras Theano conv/deconv autoencoder dimension mis-match

I'm trying to build an autoencoder in keras based on examples in this blog post but with different layers. The model compiles but throws an error (Theano) ...
user avatar
2 votes
2 answers
8k views

Sequence classification via Neural Networks

What exact kind of architecture of neural networks do I need for a sequence binary/multiclass classification? The sequences can be of different length and are to be discriminated by a certain ...
user avatar
  • 326
1 vote
0 answers
86 views

Neural network hyperparameters don't affect performance [duplicate]

I am trying to perform multi-label classification using neural network, with 3 hidden ReLU layers and a softmax output layer. I'm also using mini-batch SGD as the optimization algorithm and negative ...
user avatar
  • 175
3 votes
1 answer
2k views

How can I re-code this hierarchical model in PyMC 3?

I wish to model data from an experiment using a hierarchical Bayesian logistic regression. The experiment involved many subjects, and many trials collected from each subject. The DV is the outcome of ...
user avatar
2 votes
0 answers
577 views

Autoencoder not able to generate same input and output value

I am doing some work with Theano based auto-encoder, giving input as samples from mixture of Gaussians, one hidden layer. I expected output to be same as input, but I am not achieving it. I have been ...
user avatar
4 votes
2 answers
437 views

How are convolutional layers connected in Theano?

How are feature maps connected between two layers in Theano/Caffe/TensorFlow? For instance, if we have 32 feature maps in Conv Layer 1, and 64 feature maps in Conv Layer 2, with 64 kernels, how does ...
user avatar
1 vote
1 answer
1k views

Deep Neural Network tuning hyperparameters

I want to find the most efficient Feed forward Deep Neural Network architecture for my problem (binary classification). Study roadmap: Create network contains only one hidden layer, Tune ...
user avatar
5 votes
3 answers
3k views

Why do Deep Learning libraries force the cost function to output a scalar?

Let's say we have a neural net with: 5 input neurons some arbitrary amount of hidden layers 3 output neurons Let's say we're using minibatches of size 32. So, if we input a 5x32 matrix into the ...
user avatar
  • 313
2 votes
2 answers
4k views

Stopping Exploding Gradients in Keras

I have a LSTM (Long-Short Term Memory) Neural Network that has this structure: ...
user avatar
  • 621
0 votes
1 answer
3k views

Stacked LSTM for sequence classification - Keras - Error [closed]

Hi I am trying to use a stacked LSTM architecture in keras, similar to what is shown here https://keras.io/getting-started/sequential-model-guide/. My problem is formulated as a binary time series ...
user avatar
  • 11
1 vote
0 answers
268 views

Logistic Tensor Regression

Say there are users and they view articles and click (or not click) on articles. I represent the $i$-th user as $x_i$, a $D \times1 $ vector and $j$-th article as $z_j$, a $C \times 1$vector. The ...
user avatar
  • 123
1 vote
1 answer
785 views

Implementing backpropagation in Theano

I have been playing around with Theano for a while and read a lot of code examples and it looks like every time Theano graph needs to find gradients for a list of parameters it uses T.grad() function. ...
user avatar
5 votes
1 answer
327 views

If we have auto differentiate tool, do we also need EM algorithm?

I my opinion, EM algorithm is used to estimate the parameters of some complex log likelihood function. Because sometimes, it's hard to get the derivative, we can use EM algorithm. But if we have some ...
user avatar
  • 279
1 vote
1 answer
120 views

Commutative function mapping sequence of vectors to vector

There is sequence of vectors $[v_1, v_2, v_3]$ passed as input to deep learning regression model $F$. As observed, the order of this sequence is irrelevant, so $F([v_1, v_2, v_3]) = F([v_2, v_1, v_3])...
user avatar
1 vote
1 answer
71 views

Two ways of optimize the same function

i m reading actually this tutorial about deepLearning and in particurlar about Logistic Regression. http://deeplearning.net/tutorial/logreg.html#logreg I don't get why it first says to optimize ...
user avatar
  • 233
2 votes
0 answers
70 views

Perceptron trained on time series always predicting the same answer [duplicate]

Using the model from theano's tutorial, I'm training a 3-layers perceptron with log returns over a very large dataset (~55,000 points). The output's layer contains two neurons, one for each of the ...
user avatar
1 vote
0 answers
946 views

Simple NN with Lasagne and NoLearn

I wanted to test nolearn and Lasagne with a simple neural network and train the XOR problem, but I run into odd problems. The code below fails with the error ...
user avatar
1 vote
0 answers
105 views

How to generate n theano.shared variables for Gaussian mixture regression? [closed]

I am trying to program a Gaussian mixture regression using python, and the theano package. Suppose I have n clusters, then I need to generate n regression weight matrices for each cluster (also, n ...
user avatar
5 votes
1 answer
2k views

What is the purpose of the scaling factor used in dropout?

I have a question related to the dropout function in the LSTM tutorial: http://deeplearning.net/tutorial/code/lstm.py ...
user avatar
1 vote
1 answer
870 views

Initializing starting weights in Neural Nets with bounds?

I'm implementing a deep denoising autoencoder using Theano in Python. I know that you are supposed to initialise starting weights using random numbers and that's what Theano authors do in their ...
user avatar
2 votes
1 answer
75 views

Get confidence of net in theanets [closed]

In theanets, how can I get the net's "confidence" that it is correct? Specifically, I have two output neurons, for two classes, how can I get their values when I predict on an input?
user avatar
3 votes
3 answers
1k views

Recursive neural network implementation in Theano [closed]

Is there any available recursive neural network implementation in Theano? Theano's deeplearning.net tutorial does not present any recursive neural networks. Most Theano code I've found is CNN, LSTM, ...
user avatar
1 vote
1 answer
165 views

Code for minimalistic neural net with one hidden layer [closed]

I am new to theano and learning. Am I doing regularisation correctly? Is this term the correct L2 norm? 0.01 * (wh ** 2).sum() + 0.01 * (wy ** 2).sum() Any ...
user avatar
  • 208
4 votes
0 answers
2k views

Nesterov vs. momentum gradient descent

I implemented these two methods in a deep learning project where I am using theano. I understand the mathematical difference between these two methods, and my conceptual understanding is that ...
user avatar
  • 398
4 votes
2 answers
4k views

Robust softmax solutions for Theano?

I am implementing multilayer perceptrons with the softmax activation function over Theano. In some extreme cases I am running into problems with too high/low values in the softmax function that ...
user avatar
  • 257
3 votes
1 answer
490 views

Stackable LSTM layer trained with arbitrary BPTT time steps

Anyone knows how to make a LSTM layer that is able to be trained with arbitrary BPTT time steps and easy to be stacked together? I am now implementing a basic version of LSTM layer. My scan function ...
user avatar
  • 273
8 votes
1 answer
4k views

How do I implement a deep autoencoder?

I'm trying to replicate results of this paper using Theano. The problem at the moment is, all Theano-related tutorials are only for MNIST classifiers, which isn't much use in unsupervised image ...
user avatar
  • 181