The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [autoencoders]

Feedforward neural networks trained to reconstruct their own input. Usually one of the hidden layers is a "bottleneck", leading to encoder->decoder interpretation.

Filter by
Sorted by
Tagged with
0
votes
0answers
14 views

AutoEncoder Reconstruction error for Anomaly Detection

I'm building a convolutional autoencoder as a means of Anomaly Detection for semiconductor machine sensor data - so every wafer processed is treated like an image (rows are time series values, columns ...
1
vote
0answers
13 views

Which deep learning model to use for sequence completion

I am trying to solve the problem of sequence completion. Let's suppose we have ground truth sequence (1,2,4,7,6,8,10,12,18,20) The input to our model is an incomplete sequence. i.e (1,2,4, _ , _ ,_,...
1
vote
0answers
15 views

Enforcing constraints on weight matrices using ReLU activation

In the paper 'A Deep Non-Negative Matrix Factorization Neural Network' by Flunner and Hunter, proof of Theorem 1 says that "The ReLu Activation function is a standard approximation of a non-negative ...
1
vote
2answers
31 views

Semi-supervised objective function VAE

In Kingma's paper on Semi-supervised learning https://arxiv.org/pdf/1406.5298.pdf, we are shown equations for the ELBO for the semisupervised case, however I am having a hard trying to derive the math ...
0
votes
1answer
46 views

How is it possible for a VAE decoder to reconstruct n different classes while being limited to much lower k dimensions?

How is this even possible for the VAE Autoencoder to reconstruct n different classes while being limited to k dimensions in the ...
0
votes
0answers
33 views

Is “variational approximation” synonymous with “variational inference”?

The title says it all. I am currently reading up on deep generative models, and frequently encounter the term "variational inference" as well as the term "variational approximation" to refer to what ...
2
votes
3answers
100 views

Why is reparameterization trick necessary for variational autoencoders?

I know it is said that we do the reparameterization trick so we can do back-propagation and back-propagation cant be applied on random sampling! However, I don't precisely understand the last part. ...
-1
votes
0answers
12 views

Why is VAE a nonlinear extension of PCA (and has the “interpolation” property")?

In many research fields there is a great need of finding a PCA-like nonlinear counterpart. I know PCA has two main properties: the first is dimension reduction, it can detect the correlation between ...
1
vote
1answer
23 views

Why doesnt Conditional Variational AutoEncoders(CVAE) cluster data like the vanila VAE?

I have a hard time understanding why a Conditional VAE, doesn't cluster the data-points the way vanilla VAEs do! I was expecting to see the same or at least similar looking plots when I tried to ...
0
votes
0answers
18 views

Explain autoencoder anomalies

I developed an autoencoder model to detect anomalies in a set of signals coming from a machine. After the scoring, the most anomalous point (i.e. the ones with highest reconstruction error) are ...
0
votes
2answers
40 views

Obtaining VAE reconstruction probability

How does one calculate the reconstruction probability? Let's look at the keras example code from here. Is the reconstruction probability the output of a specific layer, or is it to be calculated ...
0
votes
1answer
85 views

Autoencoder reconstruction error threshold

I have a set of signals on which I have to implement an anomaly detection algorithm. The data is split among a reference period (i.e. last 3 months) and a test period (i.e. last week). I've already ...
0
votes
0answers
13 views

Variational auto-encoder - using reparameterization trick twice?

I'm working on the following problem (found here), where I need to implement this specific VAE: Train one VAE in this configuration on both datasets: • 2D latent variables z with a standard ...
0
votes
0answers
24 views

Can data generated by generative models be used for training unsupervised learning models? [duplicate]

I'm working on a signal denoising problem. Because of not having enough data for training I'm considering using one of the generative models like VAE or GAN to generate data similar to real data for ...
1
vote
1answer
19 views

Shouldn't we sample from the output of variational auto-encoder?

I know that the output of the VAE is the parameters of the data. For example: If the data follows normal distribution $X \sim \mathcal{N}(\mu,\sigma)$, the generative network should output $\mu$ and $\...
0
votes
1answer
24 views

Does variational auto-encoder output the variational distribution of the latent variable or the distribution of the input x?

In the simple case of mixture of gaussians(with known variance), we have 2 latent variables $\mu$ and $z$. In the vaiational auto-encoder, we assume that the model is infinite mixture of gaussians. If ...
0
votes
0answers
10 views

Probabilistic Models, what do they infer?

As per my understanding, Mixture Models such as GMM, Probabilistic Models such as Variation Autoencoder, they explain the latent space behind the features. But how they turn from learning latent space ...
0
votes
1answer
20 views

Reconstruction error drops for an anomaly?

I have a convolutional Autoencoder being used as an anomaly detector, it works well. Today however I trained it on a new training/test data set and the anomalies were exposed as a drop in ...
0
votes
1answer
58 views

Disadvantages of Mean Squared Error?

I'm using mean squared error as reconstruction error for my autoencoder. The dataset is ECG (time series) and model is conv1d. I assumed MSE as the best option for reconstruction error, but it's ...
1
vote
1answer
54 views

Variational Autoencoder, understanding this diagram

I'm not an ML scientist, but I'm trying to understand how variational autoencoder works. I'll take as reference the following diagram, which it couldn't be used for backpropagation as includes a ...
0
votes
0answers
9 views

Range of parameters for hyperparameter optmization in fully connected layers

I have designed a variational autoencoder with 2D convolutions in the encoder and decoder. I have trained this autoenocder on 50'000 unlabelled images (64 x 80). Now, I would like to use this ...
6
votes
1answer
21 views

Can I use a LSTM Autoencoder to compute similarity between two variable-length audio signals?

I would like to compute the similarity between audio signals of different length. One way of doing it is to train a RNN (LSTM/GRU) Autoencoder and extract the hidden layer representation - feature ...
2
votes
0answers
16 views

Are there autoencoders or similar neural networks that behave like generalized maps?

As per the wikipedia definition, an autoencoder consists of two maps $$\phi: \mathcal{X} \rightarrow \mathcal{F}$$ $$\psi: \mathcal{F} \rightarrow \mathcal{X}$$ such that $$\underset{\phi, \psi}{\...
0
votes
0answers
7 views

Can Autoencoder map similar objects to different places in latent space?

I have just read this blog and now I am confused by this peace of text: This is bad, because then two images of the same number (say a 2 written by different people, 2 by Alice​​ and 2 by Bob​) ...
1
vote
1answer
28 views

Does the model architecture of a CNN depend on the dimension of your input images?

By model architecture, I'm interested in knowing the following: Number of nodes in input layer Number of nodes in subsequent layers Number of layers in the architecture Number of filters and ...
1
vote
0answers
13 views

Why is the latent loss in VAEs is set to the KL divergence?

The KL Divergence is surely not the wrong way to go, but I wonder if there are any VAEs which use something like the Wasserstein Distance or even an l2 loss on the ...
1
vote
1answer
148 views

How to implement 1D Convolutional Autoencoder with multiple channels?

I want to build a 1D convolution autoencoder with 4 channels in Keras. Instead of images with RGB channels, I am working with triaxial sensor data + magnitude which calls for 4 channels. I haven't ...
0
votes
0answers
48 views

mathematics behind autoencoders

Someone knows of an article that explains in detail the mathematics behind the auto-encoders. All the articles I find show the typical diagrams ...
2
votes
1answer
122 views

Backpropagation on Variational Autoencoders

Once again, online tutorials describe in depth the statistical interpretation of Variational Autoencoders (VAE); however, I find that the implementation of this algorithm is quite different, and ...
0
votes
0answers
12 views

Generating maps from sprite indexes as one hot vectors

Goal: Use a Autoencoder to allow me generate new maps from the set of sprites from old game boy games. Old games tended to be made out of sprite/tile maps. So you can cut up their maps into 16x16 ...
0
votes
1answer
69 views

Distorted validation loss when using batch normalization in convolutional autoencoder

I have implemented an variational autoencoder with convolutional layers in Keras. I have around 40'000 training images and 4000 validation images. The images are heat maps. The encoder and decoder are ...
0
votes
0answers
37 views

Is there a Continuous Conditional Variational Autoencoder?

Conditional Variational Autoencoders (CVAE) are an extension of Variational Autoencoder (VAE). In VAEs we have no control on the data generation process, something problematic if we want to generate ...
7
votes
2answers
101 views

Why is the autoencoder decoder usually the reverse architecture as the encoder?

Every autoencoder architecture I've seen has a similar architecture, mainly that the decoder is just the reverse of the encoder. If the goal of the autoencoder is low-dimensional feature learning, why ...
0
votes
2answers
72 views

How to perform SHAP explainer on a system of models

I have developed a model with Autoencoder + XGBoost. Autoencoder is used to reduce dimensionality and then passed on to XGBoost for prediction. I would like to understand the feature importance of the ...
0
votes
1answer
64 views

Optimizing parameters for CNN autoencoder based on training and validation loss

I have designed an autoencoder with a encoder and decoder consiting of 2D convolutational layers (the input are 40'000 2D images). I train the autoencoder using adam optimizer. The autoencoders has ...
1
vote
1answer
17 views

Can variational autoencoder be used to generate similar images?

I trained the variational autoencoder . Suppose if we take the mnist dataset and visualize it, the distribution of classes are clustered but are very close to each other. When i take a point/encoding ...
1
vote
1answer
24 views

Using a separate but related dataset for feature extraction (transfer learning)

I have two datasets of MRI images: a larger one of Altzheimer's paitents (AD), which is about 3 times the size of a smaller dataset of brain tumor paitents (BT). My aim is to make use of the AD data ...
2
votes
1answer
66 views

how to interpret the sharp decline in loss in seq2seq models

I have a seq2seq model. I have applied this data over 20_newsgroup data set. My problem is that I face with exploding gradient ...
0
votes
0answers
23 views

How harmful is a wide Dense layer after a narrow?

My CNN-LSTM EEG Keras classification model includes a Dense 'shortcut' connection for residual sequence learning as shown below; to match dimensionality, the Dense layer's set to ...
1
vote
0answers
66 views

Autoencoder on heat maps

I have time sequences of 2D heat maps. For different people I have heat maps over time. For each person I have around 720 heat maps and in total around 50'000 heat maps. Now, I would like to train ...
0
votes
0answers
24 views

How can I interpret the result of get_weight of latent size in Seq2Seq model keras

My question is related to Seq2Seq models where we have LSTM as encoder and decoder. Imagine we have the Autoencoder alone, and we extract the weight associated ...
0
votes
0answers
6 views

Auto-Encoders with Sparsifying Non Linearity - sparsify function

I am currently studying for an exam and one topic are auto-encoders. The slides mention that one good encoding function could projecting the data and applying a sparsifying function, like so: $$r_i = ...
0
votes
1answer
66 views

KL divergence of a uniform prior and a custom posterior

So I was reading the Google's paper on VQ-VAE and have stumbled upon the derivation of KL divergence of the uniform prior and the given distribution: $$q(z=k \mid x)=\left\{\begin{array}{ll}{1} & ...
0
votes
1answer
45 views

Log-Normalization of skewed data before feeding to neural network models ( autoencoders)

If your input data has few columns that are extremely skewed, It is well known that one would log normalize ( take log and then normalize or standardize) the data before passing to regression ...
1
vote
1answer
104 views

Variational Autoencoder - How many Normal Distributions for Posterior

I am currently reading about variational autoencoders. Some of the papers I've read are: Tutorial on Variational Autoencoders by Doersch: https://arxiv.org/abs/1606.05908 Auto-Encoding Variational ...
0
votes
1answer
26 views

CNN architectures for predicting next frame

I am currently trying to generate image of a body tissue at time t+1, by using image given at time t (or t-1,t-2...). Until now, I experimented on some Incomplete Conv Autoencoders in two main ...
0
votes
1answer
27 views

Using an Autoencoder with uniformly distributed data

Setup: Dataset: 40k uniformly distributed 13-dim samples (floats between 0 and 1) AutoEncoder: (input: 13dim) - fc layer 13 dim, relu - latent layer - fc layer 13 dim, relu - (output: 13dim) Loss: ...
0
votes
0answers
129 views

Autoencoder loss function - why minimise MSE?

Why are most loss functions used in autoencoder learning algorithms the mean squared error?
0
votes
0answers
10 views

How to use discriminator for mapping into the same space

I have two sets of unpaired data. I use two encoders of each set to encode data points into two separate low-dimensional spaces. My goal is to make sure that points are mapped into the same space. For ...
1
vote
0answers
53 views

Latent space of VAE

I have trained a VAE on a new dataset and it has converged nicely. I am able to interpolate between samples, etc, with expected results. However, when I randomly sample the latent space with ...