Questions tagged [autoencoders]

Feedforward neural networks trained to reconstruct their own input. Usually one of the hidden layers is a "bottleneck", leading to encoder->decoder interpretation.

Filter by
Sorted by
Tagged with
0 votes
0 answers
15 views

is it good to have 100% accuracy on validation?

i'm still new in machine learning. currently i'm creating an anomaly detection for flight data. it is a multivariate time series data that include timestamp, latitude, longitude, velocity and altitude ...
user avatar
0 votes
0 answers
18 views

Advarsarial autoencoder loss function - Using MSE and BCE both

I came across this implementation of AAE on financial data to detect anomalies https://github.com/GitiHubi/deepAD/blob/master/KDD_2019_Lab.ipynb. In here for the VAE part of AAE, the author is using ...
user avatar
0 votes
0 answers
3 views

How disentaglement in latent space can produce poor variety of instances in VAE..?

I'm reading about $\beta$-VAE which essentially proposes a way to disentangle representations in the latent space. We can subjectively (I guess) identify axes carrying specific sources of variations ...
user avatar
1 vote
1 answer
59 views

Why do we need Jensen inequality for variational autoencoders?

Just to clarify, I think I understand all the derivations in context of VAEs pretty well; however, there is one last thing that I need explained. There are multiple related derivations of the evidence ...
user avatar
1 vote
1 answer
24 views

Regarding Spherical Gaussian Latent Space

I am currently reading paper on Variational Autoencoders by Dr. Kingma and Dr. Welling, and am having a few confusions. Question 1: What is a spherical Gaussian latent space? How do we interpret it ...
user avatar
  • 61
0 votes
0 answers
12 views

Understanding the essence of Reparameterization Trick in Variational Autoencoders [duplicate]

I am trying to understand the essence of reparameterization trick and for that I am following the blog post here as suggested here. However, I have a confusion in the following part: I am not being ...
user avatar
  • 61
0 votes
1 answer
14 views

Monte Carlo Gradient Estimation in Auto-encoding Variational Bayes

I am currently reading paper Auto-encoding Variational Bayes and I am not being able to understand the highlighted part in the screenshot below: I am not understanding why there is f(z) and what is ...
user avatar
  • 61
1 vote
1 answer
26 views

Prior in variational autoencoders

I am currently dealing with variational autoencoders where I've read the original paper "An introduction to variational Bayes" from Kingma and Welling. I am currently still a little confused ...
user avatar
  • 11
0 votes
0 answers
7 views

Encoding knowledge of data lying in the union of hyperplanes using differentiable optimization layers

I know that a way to possibly encode prior knowledge into neural networks training is by using differentiable optimization layers (paper). I'm in the following situation, and I'm wondering if it could ...
user avatar
1 vote
0 answers
40 views

Should I be using batchnorm and/or dropout in a VAE or GAN?

I am trying to design some generative NN models on datasets of RGB images and was debating on whether I should be using dropout and/or batch norm. Here are my thoughts (I may be completely wrong): ...
user avatar
0 votes
0 answers
16 views

Decoder distribution in avariational autoencoder

I have a little perplexity about the variational autoencoder model, by looking at some underlying terminology. We assume the approximating posterior distribution to be a Gaussian $q_\phi(\textbf{z}|\...
user avatar
2 votes
1 answer
20 views

Which exact loss do we minimize in a VAE model?

Reading about VAEs here and there, I often get stuck in the confusion about which quantity gets minimized as VAE objective. After some calculations, here's what we get at: $\log p_\theta(x) \ge - \...
user avatar
1 vote
0 answers
9 views

Are there prebuilt architectures for decoder?

I am building Variatonal Autoencoder and right now for encoder I am using EffNetB0 architecture, which is simple to implement into encoder. For decoder, I have built my own architecture with 8 CNN ...
user avatar
  • 11
2 votes
1 answer
50 views

VAE loss doesn't converge to zero. Does it make sense to sample new instances from trained latent space?

I aim to use a variational autoencoder (VAE) as a generative model. Does this make sense only if the reconstruction loss converges towards zero? On a project I'm working on, the loss is getting ...
user avatar
0 votes
0 answers
9 views

Constraining reconstructed vectors to lie in a hyperplane ina VAE

I'm trying to add a linear constraint to my variational autoencoder model. Let's say that my input is made of two concatenated vectors: $\textbf{x} = \textbf{t} \oplus \textbf{y}$ where (for example) ...
user avatar
1 vote
1 answer
23 views

Which parameters are updated in VAE with normalizing flow?

I've been reading this article about implementing a VAE with normalizing flows. What it's not clear to me, is which parameters are actually optimized using this approach. Should I only optimize the ...
user avatar
1 vote
1 answer
31 views

Using a Variational AutoEncoder with an inverse bottleneck

For a problem I'm dealing with, I'm trying to understand if my approach could make sense. I'm using a Variational AutoEncoder (VAE) having relatively low-dimensional inputs, say $x \in \mathbb{R}^n$. ...
user avatar
2 votes
0 answers
26 views

Variational vs deterministic autoencoder

I tried to use VAE and DAE to train with MNIST dataset and see their performance. It turns out VAE has a greater loss but generates better sample images. Why does it happen?
user avatar
1 vote
0 answers
28 views

Does the accuracy of an autoencoder matter when used to detect anomaly

Let me first explain what I'm trying to do: An autoencoder is first trained on regular data (no anomalies), then the decoder is discarded. Outputs from the encoder using overall data (data with and ...
user avatar
  • 143
0 votes
0 answers
38 views

Higher latent space than Input space in a Variational autoencoder (VAE)

Could it make any sense to choose a larger dimension for the latent space of the VAE with respect to the original input? For example, we may want to learn how to reconstruct a relatively low-...
user avatar
0 votes
0 answers
7 views

Can Bidirectional LSTM under perform LSTM under the exact same setting?

I've been training an LSTM autoencoder for anomaly detection on Time Series. I got some very satisfying results with this architecture and we were trying to improve the model performance. We looked at ...
user avatar
0 votes
0 answers
21 views

Compare three different algorithms for anomaly detection

I have 3 different anomaly detection algorithms, that I tested on a mock dataset of 5 elements. The output of the first and second algorithms, that implement an LSTM, is true/false according to if ...
user avatar
  • 115
2 votes
0 answers
62 views

Why do I get different results for the Isolation Forest after using PCA?

I am working on an unsupervised anomaly detection project and used the Isolation Forest and AutoEncoders (a normal one and a VAE) to detect anomalies. The AutoEncoders' prediction are identical but ...
user avatar
0 votes
0 answers
25 views

Why do we concatenate the condition vector two times in conditional variational autoencoder (CVAE)

I don't quite understand why, in conditional variational autoencoders (CVAEs) we condition on both the encoder and the decoder. In particular, in CVAE the objective function is defined to be: $$\...
user avatar
0 votes
0 answers
28 views

Help understanding variational autoencoder with learned latent structure

I’m reading and trying to understand the Variational Autoencoder with Learned latent structure (https://arxiv.org/abs/2006.10597). My understandings are they use the transport operators to define the ...
user avatar
  • 1
1 vote
0 answers
10 views

What is the mechanism that makes possible for VAEs to create meaningful concept vectors rather than purely random ones?

I don't know how to put it better, but if we have n-dimensional data and we want to compress it to m-dimensional space where m<<n what mechanism makes those m content vectors always so ...
user avatar
  • 41
0 votes
0 answers
8 views

How do you scale the activation function of an auto-encoder when using a custom normalization fitted on the data?

I'm working on a convolutional auto encoder. The input is an image The output is a reconstructed image During the training phase, we feed the same image in and out The loss is the Mean Squared Error ...
user avatar
  • 111
1 vote
1 answer
39 views

Variational Autoencoder assumtions

I am currently reading the paper "Importance Weighted Autoencoders" and am having a hard time understanding something regarding the original Variational Autoencoder (VAE) as described here ...
user avatar
0 votes
0 answers
15 views

What prevents my PyTorch convolutional auto-encoder to converge on some initializations? [duplicate]

I built a small auto-encoder for greyscale images. It is there to make some tests, so I train it often, and I have a strange behavior. On some initialisations, it does not converge. I mean, the MSE ...
user avatar
  • 111
1 vote
1 answer
44 views

Clarification on normal distribution assumption for probablistic decoder P(x|z) in VAE

After searching around on the internet, it seems that the following assumption seems to be commonly used in VAEs: For a continuous domain (MNIST), assume $p(x|z) = N(x; f(z), \sigma^2)$. Where $f(z)$ ...
user avatar
0 votes
0 answers
14 views

Validation error saturates

I am training Autoencoder (1 hl) on keras. I have used validation split of 20%. Training seems ok, but for the validation set the MSE error(shown in blue) seems to saturate with whatever no. of hidden ...
user avatar
1 vote
1 answer
43 views

prepare data for autoencoder

I have for example 500 engines and for each engine I have 100 features and each feature is measured 1000 times. It means the data table is: 500,000 rows and 100 columns. (for each engine I have only a ...
user avatar
  • 113
0 votes
0 answers
30 views

LSTM Autoencoder for online anomaly detection

I would like to use an LSTM-Autoencoder for an anomaly detection task, but in an online setting, meaning we are observing the data as it is streaming in. What I would like to do is given some discrete ...
user avatar
0 votes
0 answers
40 views

What pretrained word embeddings does the Universal sentence encoder use for Deep Averaging Network?

The paper for Universal sentence encoder Universal sentence encoder paper! is pretty straightforward, and so is the paper for Deep averaging network Deep averaging network paper! but I'm confused ...
user avatar
  • 13
0 votes
0 answers
8 views

Conventional autoencoder training instability [duplicate]

I am currently writing an autoencoder in python (torch); its encoder is intended to serve as a compression tool. The input dataset contains a mix of numerical data (including large integers), ...
user avatar
5 votes
1 answer
788 views

Implementing a VAE in pytorch - extremely negative training loss

So we can derive the loss function for the VAE following something like this: https://arxiv.org/pdf/1907.08956v1.pdf But when I go to implement the loss function in pytorch using the negative log-...
user avatar
1 vote
0 answers
23 views

Implementing a VAE in pytorch - negative loss [closed]

So we can derive the loss function for the VAE following something like this: https://arxiv.org/pdf/1907.08956v1.pdf But when I go to implement the loss function in pytorch, with MSE as the ...
user avatar
4 votes
1 answer
82 views

Autoencoder doesn't learn 'sparse' input images

I am trying to train an autoencoder with PyTorch on 2D images containing 2D Gaussian densities such as this: The images are of size 100x100 (I feed them into the autoencoder as 1x10000 tensors). The ...
user avatar
0 votes
0 answers
71 views

Triplet loss for text embedding and text similarity? [duplicate]

I am working on a triplet loss based model for text embedding. Short description: I have a database about online shop, I need to find the suitble product when users enter a text on search bar. I ...
user avatar
1 vote
0 answers
7 views

Associate prediction task to a graph autoencoder (GAE)

I have been reading about graph autoencoders and was wondering if it might be a reasonable idea trying to associate to the unsupervised setting which produces some best low-dimensional representation ...
user avatar
0 votes
0 answers
42 views

Variational inference - posterior predictive distribution

So suppose we have a neural network that aims to map values from one distribution to another. That is to say the inputs do not belong to the same distribution as the targets. It then follows that, ...
user avatar
0 votes
0 answers
15 views

Bottleneck Layer VAE - values centred around 0 with small standard deviations

As the title states, I'm curious as possible theoretical explanations for why might VAE bottleneck layer have values more or less centred around 0 with small standard deviations. I have found this out ...
user avatar
0 votes
0 answers
21 views

Choice of Size of Latent Representations in Variational Autoencoders

Are there any guidelines concerning the choice of the size of the latent space when using VAEs? It does of course depend on the data. I found the following in Christopher Bishop's Book Pattern ...
user avatar
0 votes
0 answers
86 views

Loss Function of a Variational Autoencoder when using Implicit Reparameterization Gradients (Dirichlet distributed latent space)

I would like to implement a VAE with a Dirichlet distributed latent space in Python. Since the reparametrization trick does not work for the Dirichlet Distribution I would use Implicit ...
user avatar
0 votes
0 answers
79 views

Variational Autoencoder with Dirichlet distributed latent space using the Weibull Distribution

My goal is to create an VAE with an Dirichlet distributed latent space. Since the reparametrization trick does not work for the Dirichlet Distribution, I am trying to approximate the Gamma ...
user avatar
2 votes
0 answers
22 views

Which part of an encoder/decoder generative network is improved by adding a discriminator loss term?

Lets say you're doing a superresolution image task with "deep learning" constructs. You encode to a latent representation using some parameterized model (like a neural network), then decode ...
user avatar
  • 141
0 votes
0 answers
92 views

Are convolutional autoencoders required to have symmetric encoders and decoders?

I am a newer to deep learning. Recently I am studying the convolutional autoencoder (CAE). I found the architectures built with keras and matlab are a little different. In particular, the architecture ...
user avatar
0 votes
0 answers
54 views

Does it make sense to use the decoder of LSTM autoencoder as an input to K-means clustering?

I have a large time-series data - with 200 (time) variables. I want to use a clustering algorithm to cluster my data based on the values of (200) time variables. Before using a clustering algorithm ...
user avatar
  • 123
1 vote
0 answers
59 views

How to choose the number of latent dimensions in VAE?

I have trained a VAE that can generate photos of human faces. I have isolated the dimension that correlates most to smiling and now I only want the VAE to generate smiling faces. May I know is it a ...
user avatar
5 votes
1 answer
44 views

Why in some neural network training cases the outputs are assumed to be a probability distribution?

This might be a stupid question but this question is bugging me for a long time. When I first started working with neural networks we usually created a neural network which output vector of number ...
user avatar
  • 2,031

1
2 3 4 5
12