Skip to main content

Questions tagged [torch]

Scientific computing framework for LuaJIT.

Filter by
Sorted by
Tagged with
1 vote
0 answers
36 views

Reproducing results from classic dropout paper [closed]

In the classic paper "Dropout: A Simple Way to Prevent Neural Networks from Overfitting", there is a figure comparing the features learned by a one-layer autoencoder trained on MNIST with ...
Ari Herman's user avatar
0 votes
0 answers
100 views

Is my custom loss function differentiable?

Consider the following loss function. loss = ( ( torch.where(d > threshold, torch.sqrt(d), 0) * t ) + ( torch.where(d <= threshold, (1 - d), 0) * (1 - t) ) ) ...
Adel's user avatar
  • 295
0 votes
0 answers
32 views

How to Change Architecture of DCGans?

https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html I was refering this notebook but default size is 64*64 I want to change architecture to 256 or 512 Can anyone help me with training ...
user20332627's user avatar
1 vote
0 answers
12 views

CNN: adding a third class improves the overall ranking of the network

I am working on a Convolutional Neural Network for classifying two classes of images whose difference between them is very small. Running the CNN (using PyTorch), it was able to correctly classify ...
donut's user avatar
  • 263
1 vote
0 answers
29 views

VAE: Noisy decoder output

I'm trying to implement a simple VAE by following several tutorials like this and this. This is the code that I came up with: ...
JosephM's user avatar
  • 11
0 votes
0 answers
20 views

GoogleNet-LSTM, cross entropy loss does not decrease [duplicate]

...
makala's user avatar
  • 11
1 vote
0 answers
250 views

Graph Convolutional Network work badly: my accuracy doesn’t grow!

I’m trying to classify some drugs like very active, active, non active (label: 0, 1, 2) against the cancer. To do that I built a Graph Convolutional Network using PyTorch Geometric, this is the code: <...
Gianmarco Luchetti 's user avatar
1 vote
0 answers
863 views

How do you use pytorch to solve strictly constrained optimization problems? [closed]

I am trying to solve the following problem using pytorch: given a six sided die whose average roll is known to be 4.5, what is the maximum entropy distribution for the faces? (Note: I know a bunch of ...
Paul Siegel's user avatar
1 vote
0 answers
33 views

Am I need to differentiate expected value and variance in BatchNorm backward implementation?

I'm implementing BatchNorm layer now. I use Module as base class, but change some methods -- for forward-pass and backward-pass: ...
taciturno's user avatar
  • 251
8 votes
1 answer
4k views

Why does torchvision.models.resnet18 not use softmax?

I see image-classification models from torchvision package don't have a softmax layer as final layer. For instance, the following snippet easily shows that the resnet18 output doesn't have a sum = 1, ...
Luigi D'Amico's user avatar
3 votes
1 answer
593 views

I can't overfit a simple (linear) dataset - PyTorch

I have created a small Pytorch template to try to overfit a small dataset (of only ten points) by using linear regression with polynomial features. The method works by using gradient descent. In ...
Ambesh's user avatar
  • 333
4 votes
1 answer
2k views

How to force logistic regression weights to be always positive in pytorch? (equivalent of keras NonNeg in pytorch) [closed]

I am solving a binary classification task, and I need my logistic regression's learned weights to be all positive. This is my current classifier implemented in pytorch : ...
OneAndOnly's user avatar
1 vote
1 answer
2k views

Graph convolution network for variable number of nodes

Is it possible to train a graph convolutional network on graphs with a varying number of nodes? I have a dataset of graphs with a range of 400-1000 nodes, though I could see a higher number of nodes ...
rmaguiar's user avatar
  • 113
4 votes
1 answer
1k views

he_normal (Keras) is truncated when kaiming_normal_ (pytorch) is not

Thanks for having a look at my post. I had an extensive look at the difference in weight initialization between pytorch and Keras, and it appears that the definition of he_normal (Keras) and ...
londumas's user avatar
2 votes
1 answer
2k views

How exactly does conv1d filter work when operating on a sequence of characters?

I understand convolution filters when applied to an image (e.g. an 224x224 image with 3 in-channels transformed by 56 total filters of 5x5 conv to a 224x224 image with 56 out-channels). The key is ...
Joe Black's user avatar
  • 319
1 vote
1 answer
319 views

Outliers Kalman Filtering

This might not be the right place to ask this questions, but I figured it's more of a machine learning question. I am also asking on the pyro forum for brevity. I'm working with the simple extended ...
sjdgg93's user avatar
  • 21
1 vote
1 answer
1k views

Increasing image size in pytorch celebrity generating GAN? [closed]

complete newbie here, bear with me. I'm making my way through this tutorial: https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html Upon attempting to make a simple change to the image ...
a meme's user avatar
  • 11
2 votes
0 answers
89 views

When to stop training a Auixilary Classifier GAN?

When should I stop training an Auxiliary Classifier GAN? when the discriminator loss converges or when the auxiliary classification accuracy converges or when the Real vs Fake classifier converges to ...
user570593's user avatar
  • 1,119
5 votes
1 answer
2k views

Pytorch Cross Entropy Loss implementation counterintuitive

there is something I don't understand in the PyTorch implementation of Cross Entropy Loss. As far as I understand, theoretical Cross Entropy Loss is taking log-softmax probabilities and output a real ...
Julien Rhapsodos Girard's user avatar
0 votes
1 answer
71 views

RNN not matching expected output

I am trying to build an RNN to predict a time-series signal based on knowledge of 3 others that I believe to be related to the output. I am using the rnn Lua Torch ...
beldaz's user avatar
  • 135
5 votes
1 answer
6k views

Sigmoid activation hurts training a NN on pyTorch

I'm a beginner in the field of Machine Learning and I'm currently trying to get my hands "dirty" for the first time with some code after completing a course in that field. I'm using pyTorch to train ...
Mickey's user avatar
  • 213
12 votes
1 answer
12k views

Dynamic graphs versus static graphs in deep learning libraries

What is the difference between dynamic graphs and static graphs in deep learning libraries? Which one is faster? and when to use each one of them?
floyd's user avatar
  • 1,382
2 votes
1 answer
379 views

Weight matrices computation in attention-based encoder of Deep Learning NLP

On page 4 of this research paper titled A Neural Attention Model for Sentence Summarization , it is mentioned the attention-based encoder is determined by the ...
Mr_RexZ's user avatar
  • 143
1 vote
0 answers
50 views

Prediction with large inputs and outputs

Being a total newbie with machine learning, I thought I would seek your advice on a problem of mine. I'm looking for any leads, starting points ans help you could provide me. I'm looking to predict ...
Hetana's user avatar
  • 11
1 vote
1 answer
289 views

How to use torch RNN and LSTM packages and are they necessary?

I have come across this brilliant site where I finally understood BPTT for RNN's and want to implement it. The code is given in python but I want to implement it in torch using lua. I have understood ...
RaviTej310's user avatar
1 vote
1 answer
4k views

Loss functions that act on real-valued output vectors (and NOT just on 1-hot vectors)

I am trying to modify Andrej Karpathy's char-RNN code. As far as I understand, the loss function used in his code for a LSTM is the Softmax function function (in the file model/LSTM.lua ). I ...
Iceflame007's user avatar
1 vote
2 answers
977 views

How to train LSTM for a simplest function recognition

I'm learning LSTM networks and decided to try synthetic test. I want LSTM network fed by some points (x,y) to distinguish between three basic functions: line: y = k*x + b parabola: y = k*x^2 + b sqrt:...
Pavel Chernov's user avatar
1 vote
1 answer
7k views

In neural networks, how to compute the mean square error (MSE) in gradient update when using a minibatch?

I've been using a siamese neural network for the binary classification of biological data. I've implemented a Torch version of this algorithm, including a stochastic gradient update function. At ...
DavideChicco.it's user avatar
2 votes
0 answers
305 views

Difficulty learning parameters in RNN?

I'm implementing an LSTM using the RNN package in Torch. I've been able to get very simple models to converge (like learning the relation f(x) = x), but haven't been able to get basic things like ...
user317258's user avatar
12 votes
2 answers
10k views

How does the DepthConcat operation in 'Going deeper with convolutions' work?

Reading Going deeper with convolutions I came across a DepthConcat layer, a building block of the proposed inception modules, which combines the output of multiple tensors of varying size. The authors ...
moi's user avatar
  • 225