Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [autoencoders]

Feedforward neural networks trained to reconstruct their own input. Usually one of the hidden layers is a "bottleneck", leading to encoder->decoder interpretation.

0
votes
0answers
7 views

What is the best input for denoise autoencoder for sound/audio data?

I am currently trying to build an autoencoder to de-noise audio data. However I have not found any good articles explaining about the input to the autoencoder, i.e. feature vector. As in speech ...
0
votes
1answer
37 views

When does my unsupervised autoencoder start to overfit?

I am working on anomaly detection using an autoencoder neural network with $1$ hidden layer. This is an unsupervised setting, as I do not have previous examples of anomalies. The input data has ...
0
votes
0answers
11 views

What's the difference between random and deterministic encoder in autoencoders?

I read this paper "Wasserstein Auto-Encoders", and they mention deterministic encoder and random encoder but without stating the difference between them. How can we tell the difference?
0
votes
0answers
23 views

Autoencoder as an optimization (search) problem

We all know that machine learning problems can be modeled as an optimization problem where we are searching for the best set of parameter values in the parameter space that optimizes our objective ...
0
votes
1answer
30 views

Help in Understanding Variational Autoencoder Size of Latent Variables

I'm trying to understand further how a variational autoencoder works beyond the conceptual level. However I'm still confused as to what the "vector of mean and variances" can look like and to digest ...
1
vote
0answers
24 views

Equity Risk Model using an autoencoder

I am trying to create a statistical equity risk model using an autoencoder in a similar fashion to how one would use PCA to derive the systematic and specific risk components of a stock's returns. I ...
0
votes
1answer
21 views

Minimum Input Dimension for Autoencoder Neural Network

Model: Assume we want to learn patterns using an autoencoder neural network. In the simplest case, such a network is "shallow" with 1 hidden layer, takes a $d$-dimensional numerical input vector $x$, ...
0
votes
0answers
16 views

Correct numerical feature transformation for neural networks

Model: I am working on a "shallow" (3-layer) auto encoder neural network. The input layer receives a, say 25-dimensional, vector $x$ of numerical elements representing client purchases. Several ...
1
vote
1answer
17 views

Prerequisites for Wasserstein GAN/Autoencoder

Can someone who read WGAN/WAE papers and understood Wasserstein part, could you share how you prepared necessary Optimal Transport background? The mentioned papers seem little tough if you don't have ...
0
votes
0answers
10 views

What are the shortcomings of calculating the loss in pixel space vs. feature space

While training (Variational)-Autoencoder networks, I came along the paper by Higgins et al. "DARLA" where she stated: The shortcomings of calculating the log-likelihood term [...] on a per-pixel ...
1
vote
1answer
59 views

Generative autoencoders - how important is agreement of latent variable distribution e.g. with Gaussian?

Autoencoders want to minimize distortion of encoding-decoding process, preferably alongside evaluation by discriminant. Generative autoencoders additionally would like latent variable from a chosen ...
1
vote
1answer
32 views

DeepLearning & Anomaly Detection - Understanding & How to Properly Tune

I'm looking into understanding the Deeplearning anomaly detection algorithm provided by h2o. I tried to recreate an example below. Perhaps some of these questions are basic, but I'm trying to better ...
0
votes
0answers
24 views

Categorical data & Gaussian latent variables

I am learning about imposing structure on the latent variables in autoencoders. In that context I have looked at variational autoencoders (VAEs) and adversarial autoencoders (AAEs). This paper ...
0
votes
1answer
56 views

VAE with mixture of gaussian prior

I try to understand this paper where they try to use a mixture of Gaussian as a prior, instead of the standard gaussian. There are several things unclear to me though: They say that they set $\pi_k = ...
1
vote
1answer
40 views

Understand VAE with VamPrior

I am currently reading this paper. The authors propose to use as an prior this expression: $$ p_\lambda(z) = \frac{1}{K} \sum^K_{k=1} q_\phi (z\mid u_k) $$ where $q_\phi$ is the encoder, and $u_k$ is ...
1
vote
0answers
16 views

Use internal representation of autoencoder for anomaly detection [closed]

I've trained an autoencoder to recognize 'positive' time series (the network is a simple fully connected network, no recurrent layers). The problem is that from what my advisor says, I should try to ...
0
votes
0answers
18 views

Forming Conditional Probabilities in tensorflow

This is a repost from stack, but I think it may be more applicable here. Granted there is even a bounty for it there, if someone could answer it. https://stackoverflow.com/questions/53421179/forming-...
2
votes
1answer
29 views

KL-Diverence of Q(z|X) and P(z) in Variational Autoencoder (VAE)

I aim to understand how $D_{KL}[Q(z | X) || P(z)]$ can be converted to $\frac{1}{2} \sum_{k} (\Sigma(X) + \mu^{2}(X) - 1 - \log \Sigma(X))$, where $k$ is the dimension of the Gaussian distribution. ...
0
votes
1answer
27 views

Implementation of WAE-GAN does not match with the description in the paper

According to the litterature and specifically to this paper, the wasserstein autoencoders is an encoder-decoder architecture. So it must contain encoder and decoder parts. in the algorithm ...
3
votes
2answers
53 views

Encoder Decoder networks with varying image sizes

Encoder Decoder Network - Computerphile : At the very beginning of this video, Michael Pound goes on to say: So it (encoder decoder network) makes no assumptions about the size of the input the ...
0
votes
0answers
19 views

Can attention be implemented without encoder / decoder?

I just got into models beyond biLSTM, would like to start with applying attention to my existing network (RNN). I find examples for attention always with encoder decoder architecture, however is it ...
-1
votes
0answers
53 views

Reconstruction loss for variational auto-encoders

In the original paper of variational auto-encoder (VAE), the estimated lower bound is as follows where the negative of the second term is reconstruction loss. For the case of Gaussian decoder(...
0
votes
2answers
90 views

convolutional autoencoder on an odd size image

I am trying to apply convolutional autoencdeor on a odd size image. Below is the code: ...
1
vote
1answer
22 views

How can I tell if an Autoencoder is encoding my data properly?

Autoencoders can be classified as a method of unsupervised learning, and unsupervised learning often comes with a problem where it's hard to tell if the model is working properly. However, some ...
1
vote
1answer
46 views

Using step function as activation function in the final layer

I am using variational autoencoders as machine learning algorithm. My input data are images/matrices that represent user interface layouts or how the HTML page will be divided. I am thinking to ...
0
votes
0answers
62 views

Autoencoder with small dataset - simple images

I wanted to ask a simple question regarding autoencoders to parse for tips and possible advice before diving into this path. I have a small dataset ~50-100 heatmap images that are relatively simple. ...
0
votes
1answer
42 views

back propagation in neurons with zero weight and some specific conditions

I have read a lot of articles to understand what is happening behind the scene in backpropagation like Ive gone through this and many other like that. I think I understand how the backpropagation ...
1
vote
0answers
54 views

How to evaluate performance of (variational) autoencoders?

Let's assume that you have trained your (variational) autoencoder on MNIST digits. After some time, you check the result and decide that the reconstruction is pretty good. But this is highly ...
0
votes
0answers
9 views

How to assign labels to un-labeled documents

I have a bunch of text documents and they are not labeled but each document represents one or more than one category/label. I'd like to assign the appropriate label (s) to each document. What's the ...
0
votes
1answer
154 views

Discrete Random Variables and Deep Generative Models - Why Gumbel-Softmax is needed?

I am reading this 2014 NIPS paper on deep generative models and their application to latent discrete random variables, and this 2017 ICLR paper on Gumbel-Softmax. I essentially don't understand why we ...
0
votes
1answer
20 views

Using the Expected value of the log as a score for the anomaly detection instead of just the expected value

While dealing with anomaly detection using a probabilistic model I need to compute the probability of an example coming out of the model I built. More specifically: If $p(X)$ is the model I built and ...
0
votes
0answers
38 views

Training an autoencoder with reconstruction target and classification labels

I would like to binarily classify a number of sequences which contain heavy noise. For each of these sequences, I have another sequence which is related to it, contains less noise and is also closely ...
0
votes
0answers
25 views

Is z-score normalization with hard caps reasonable?

I am currently trying to train an variational autoencoder that is implemented in TensorFlow. I have a training set with around 25 000 samples which I normalize by using the formula where the $i$ ...
1
vote
1answer
268 views

“Denoising” autoencoder with distortions other than Gaussian noise

I watched some talks by Yoshua Bengio, where he often refers to denoising auto-encoders (AE) as a powerful method to learn (unsupervised) representations on an input space (e.g. here). The idea as ...
2
votes
1answer
256 views

Why binary crossentropy can be used as the loss function in autoencoders?

I was wondering why binary crossentropy can be used as the loss function in autoencoders trained on (normalized) images, e.g. here or this paper? I know that binary crossentropy can be used in binray ...
0
votes
0answers
24 views

Variational Auto-encoder for supervised learning

It seems that variational auto-encoders (VAE) has become one of the most popular technique for generative modeling. However, is it possible to use variational auto encoders for discriminative ...
2
votes
1answer
40 views

Gaussian log-density variational derivative

In Appendix C.1 of 'Taming VAEs' paper, the authors need to compute the functional derivative $$\frac{\delta}{\delta g\left( z \right)} \mathbb{E}_{q\left(z\mid x\right)} \left[(g \left( z \right) - ...
0
votes
0answers
16 views

Denoising Autoencoder Parameter Search

I have ran a hyperparameter search for a denoising autoencoder and the results suggest I should make the sizes of my hidden layers as large as possible (within the range of values I allowed it to ...
3
votes
1answer
61 views

When generating samples using variational autoencoder, we decode samples from $N(0,1)$ instead of $\mu + \sigma N(0,1)$

Context: I'm trying to understand the use of variational autoencoders as generators. My understanding: During training, for an input point $x_i$ we want to learn latent $\mu_i$ and $\sigma_i$ and ...
1
vote
1answer
229 views

AutoEncoders and linear activation output function

This is not a duplicate of the Activation functions for autoencoder performing regression because there is a comment that somebody found a linear activation function but: they never said what it was. ...
1
vote
1answer
134 views

Can I use ReLU in autoencoder working with negative inputs?

I am using autoencoders to compress data and I see all examples on the internet using ReLU activation with image datasets. However, I am planning to use a dataset that has negative values and I was ...
2
votes
2answers
232 views

Reparameterization trick for gamma distribution

I am reading the work of Welling on Vartiational Auto-Encoders (VAE), and wonder if there is any way to generate Gamma distributed samples via a similar reparametrization? The idea of ...
0
votes
1answer
125 views

Is the output of a variational autoencoder meant to be a distribution that can be sampled, or a sample directly?

It is difficult to ask this question succinctly in the title, so let me explain. From all the examples of VAEs I have seen, there seem to be 2 approaches used to implement them. In these, ...
4
votes
1answer
217 views

Why do we need the temperature in Gumbel-Softmax trick?

Assuming a discrete variable $z_j$ with unnormalized probability $\alpha_j$, one way to sample is to apply argmax(softmax($\alpha_j$)), another is to do the Gumbel trick argmax($\log\alpha_j+g_j$) ...
1
vote
0answers
9 views

Adding Denoiser to Existing Autoencoder Network

I currently have implemented a Machine Learning Model that is very similar to the model proposed by this paper. It works pretty well on the data I have, although sometimes the training goes awry and I ...
1
vote
0answers
135 views

How to implement custom loss function on keras for VAE

I have implemented a custom loss function. While training the model, I want this loss function to be calculated per batch. ...
0
votes
0answers
124 views

How to choose suitable Autoencoder (LSTM) architecture?

I am new to Autoencoders and I am a bit confused on which model to try for my situation and what is the difference between all the different models I have seen in tutorials. So, I have a set of time-...
1
vote
0answers
22 views

Long repetitive output after changing vocabulary in seq2seq model

I trained a neural question generation model, which produces sensible questions for the vocabulary that they distributed with the paper. I wanted to run the model on a different set of word embeddings ...
0
votes
0answers
17 views

Data Scaling: is multi-dimension scaling equivalent to uni-dimension scaling for same range features?

I am given an $n \times m$ matrix $\mathbf R$ where $n$ is the number of $users$ and $m$ is the number of $items$ - this matrix is usually known as the rating matrix within recommender systems domain. ...
0
votes
0answers
34 views

Where's wrong in my reasoning behind upper bound for reconstruction error?

In the paper Mutual Information Neural Estimation, the authors derive the reconstruction error in BiGAN as $$ \mathcal R=E_{x\sim q(x)}E_{z\sim q(z|x)}\left[-\log p(x|z)\right] $$ where $q(z|x)$ is ...