Questions tagged [autoencoders]

Feedforward neural networks trained to reconstruct their own input. Usually one of the hidden layers is a "bottleneck", leading to encoder->decoder interpretation.

Filter by
Sorted by
Tagged with
0
votes
1answer
19 views

Optimizing parameters for CNN autoencoder based on training and validation loss

I have designed an autoencoder with a encoder and decoder consiting of 2D convolutational layers (the input are 40'000 2D images). I train the autoencoder using adam optimizer. The autoencoders has ...
1
vote
1answer
12 views

Can variational autoencoder be used to generate similar images?

I trained the variational autoencoder . Suppose if we take the mnist dataset and visualize it, the distribution of classes are clustered but are very close to each other. When i take a point/encoding ...
0
votes
0answers
9 views

different way of preparing text data for LSTM model

I am working on the lstm autoencoder models. Since then, I have seen two approach to prepare the text data to feed to model. word embedding like w2vec, glovec one-hot encoding What are other option ...
1
vote
1answer
20 views

Using a separate but related dataset for feature extraction (transfer learning)

I have two datasets of MRI images: a larger one of Altzheimer's paitents (AD), which is about 3 times the size of a smaller dataset of brain tumor paitents (BT). My aim is to make use of the AD data ...
2
votes
1answer
51 views

how to interpret the sharp decline in loss in seq2seq models

I have a seq2seq model. I have applied this data over 20_newsgroup data set. My problem is that I face with exploding gradient ...
0
votes
0answers
8 views

Variational autoencoder giving me nan [closed]

I have gone through my code on variational autoencoder to know why it is giving me nan and I have tried checking my data to see if there any nan in it but there non at all. I still could not ...
0
votes
0answers
13 views

How harmful is a wide Dense layer after a narrow?

My CNN-LSTM EEG Keras classification model includes a Dense 'shortcut' connection for residual sequence learning as shown below; to match dimensionality, the Dense layer's set to ...
1
vote
0answers
38 views

Autoencoder on heat maps

I have time sequences of 2D heat maps. For different people I have heat maps over time. For each person I have around 720 heat maps and in total around 50'000 heat maps. Now, I would like to train ...
0
votes
0answers
19 views

How can I interpret the result of get_weight of latent size in Seq2Seq model keras

My question is related to Seq2Seq models where we have LSTM as encoder and decoder. Imagine we have the Autoencoder alone, and we extract the weight associated ...
0
votes
0answers
6 views

Auto-Encoders with Sparsifying Non Linearity - sparsify function

I am currently studying for an exam and one topic are auto-encoders. The slides mention that one good encoding function could projecting the data and applying a sparsifying function, like so: $$r_i = ...
0
votes
1answer
37 views

KL divergence of a uniform prior and a custom posterior

So I was reading the Google's paper on VQ-VAE and have stumbled upon the derivation of KL divergence of the uniform prior and the given distribution: $$q(z=k \mid x)=\left\{\begin{array}{ll}{1} & ...
0
votes
0answers
10 views

Log-Normalization of skewed data before feeding to neural network models ( autoencoders)

If your input data has few columns that are extremely skewed, It is well known that one would log normalize ( take log and then normalize or standardize) the data before passing to regression ...
1
vote
1answer
61 views

Variational Autoencoder - How many Normal Distributions for Posterior

I am currently reading about variational autoencoders. Some of the papers I've read are: Tutorial on Variational Autoencoders by Doersch: https://arxiv.org/abs/1606.05908 Auto-Encoding Variational ...
0
votes
1answer
21 views

CNN architectures for predicting next frame

I am currently trying to generate image of a body tissue at time t+1, by using image given at time t (or t-1,t-2...). Until now, I experimented on some Incomplete Conv Autoencoders in two main ...
0
votes
1answer
17 views

Using an Autoencoder with uniformly distributed data

Setup: Dataset: 40k uniformly distributed 13-dim samples (floats between 0 and 1) AutoEncoder: (input: 13dim) - fc layer 13 dim, relu - latent layer - fc layer 13 dim, relu - (output: 13dim) Loss: ...
0
votes
0answers
30 views

Autoencoder loss function - why minimise MSE?

Why are most loss functions used in autoencoder learning algorithms the mean squared error?
0
votes
0answers
9 views

How to use discriminator for mapping into the same space

I have two sets of unpaired data. I use two encoders of each set to encode data points into two separate low-dimensional spaces. My goal is to make sure that points are mapped into the same space. For ...
1
vote
0answers
30 views

Latent space of VAE

I have trained a VAE on a new dataset and it has converged nicely. I am able to interpolate between samples, etc, with expected results. However, when I randomly sample the latent space with ...
0
votes
0answers
14 views

Maximize ELBO in Keras

When we train a Variational Autoencoder we say that we want to maximize the ELBO. However, from the Keras documentation, it seems that we are actually minimizing the ELBO: ...
1
vote
1answer
35 views

Can i use autoencoder for predicting time series missing data?

I have time series data set of current and voltage at a regular interval of time there are some missing value . can i use autoencoder to predict the missing value?
0
votes
0answers
24 views

Can this be simplified $\mathbb{E}_{q(\vec{z} \mid \vec{x})}\left[ \log {p(\vec{x} \mid \vec{z})}\right]$?

Assume that $p$ and $q$ are two distributions and $x$ and $z$ are two random variables. Can the following term (which appears in the paper Auto-Encoding Variational Bayes) be further simplified? $$\...
2
votes
1answer
58 views

Why is random sampling a non-differentiable operation?

This answer states that we cannot back-propagate through a random node. So, in the case of VAEs, you have the reparametrisation trick, which shifts the source of randomness to another variable ...
0
votes
0answers
17 views

what is the difference between a multilayered autoencoder and a hierarchical latent variable model?

I have been trying to understand how hierarchical latent variable models are different from multilayered autoencoders and in specific the argument below Autoencoder networks resemble in many ways ...
0
votes
0answers
43 views

What is the encoding in variational autoencoders?

The question is actually less broad than it sounds. I generally do understand how variational autoencoders work. From the encoding step we get four parts: mean $\mu$ standard deviation $\sigma$ ...
0
votes
0answers
34 views

LSTM Autoencoders - architecture

I am a bit confused about the structure of LSTM autoencoders, as far as I know, common way to construct vanilla autoencoders is bottleneck structure, for instance, start with 40 nodes, encode it to 30 ...
0
votes
0answers
73 views

why is VAE reconstruction loss equal to MSE loss

At which situations does reconstruction loss of VAE equals MSE loss between input and reconstructed output? Other answers where not complete!
1
vote
1answer
74 views

VAE: why we do not sample again after decoding and before reconstruction loss?

In many of the VAE schematics and in the original paper, a sampling step is present after decoding and before the reconstruction loss as shown in the image below. The image comes from Stanford CS321n. ...
2
votes
1answer
59 views

Why aren't auto-encoders also considered generative models?

Auto-encoders (AEs) are composed of an encoder and a decoder (often represented by a neural network). The encoder produces a vector representation $z$ of its input $x$ (e.g. an image). The decoder ...
2
votes
0answers
53 views

Where does the prior distribution $p(z)$ for adversarial autoencoders come from?

I am trying to understand how an adversarial autoencoder works. The discriminator takes as an input the aggregated posterior $q(z)$ generated from the decoder and matches this against the prior ...
2
votes
1answer
57 views

What is the relation between ELBO and SGVB?

Evidence lower bound (ELBO) can be minimised, so that to find the most appropriate approximative distribution of the target distribution, which is equivalent to the maximisation of the corresponding ...
0
votes
1answer
26 views

What is meant by Low-Order combination of features?

I came across a Machine Learning paper that talks about input with low-order combination of features. A statement says: The initial feature is used as the input of the model, and the non-linear ...
1
vote
1answer
41 views

The best number of nodes in bottleneck layer in Autoencoder

I would like to perform dimensionality reduction using autoencoders (similar to PCA) and I am not sure how many components are optimal i.e. what should be the size of the bottleneck layer. I was ...
0
votes
0answers
42 views

Dealing with images of variable resolution in CNN autoencoders

Let's suppose would like to build a CNN autoencoder that would be able to turn greyscale images into coloured ones. The final model should be able to accept images of any resolution. Also, note that ...
0
votes
1answer
90 views

Correct way to calculate MSE for autoencoders with batch-training

Suppose you have a network representing an autoencoder (AE). Let's assume it has 90 inputs/outputs. I want to batch-train it with batches of size 100. I will denote my input with ...
1
vote
0answers
33 views

Can you use VAEs to produce deep word embeddings?

There are many articles about applications of VAE such as image reconstruction, denoising, data compression / augmentation. However, I have not seen an example of embeddings for high dimensional data ...
0
votes
0answers
14 views

How to train the deep neural network efficiently when the input data are unstructured

Background So the background is that I want to use a deep neural network to model a system. In a traditional way to observe the system character, we will use the Gaussian noise as the inputs of the ...
0
votes
0answers
73 views

Convolutional Conditional Variational Autoencoder Implementation

This may be a rather trivial question, but I am somewhat confused. I have been able to implement a convolutional variational autoencoder. I have also been able to implement a conditional variational ...
0
votes
0answers
53 views

Many-hot encoding for Hamming codes using 1D convolutional autoencoder

I am trying a simple experiment where I take 16 numbers [0 to 15] that represent the Hamming codebook, and try to reconstruct it using an autoencoder. Instead of using a one-hot encoding scheme (16 ...
0
votes
0answers
10 views

Autoencoder keeping constant vector as predict in keras [duplicate]

I'm new in keras and deep learning field. In fact, I want to make a dense vector for each document in my data so that i built a simple autoencoder using keras library. The input data are normalized ...
2
votes
1answer
29 views

Do fully connected layers in the middle of a network impede optimization?

I submitted a paper that uses an auto-encoder network with several convolutional layers in both the encoder and the decoder and a fully connected layer (FCL) in between. Besides the FCL being useful ...
0
votes
0answers
11 views

Autoencoder - reconstructed image not matching the input image

I have trained a convolutional autoencoder on cifar10 dataset. The reconstruction loss on the test data is quite less (around 0.0225). However, the reconstructed training images do not look like ...
1
vote
1answer
29 views

How is the standard deviation of VAE's constructed?

I am trying to build a Variational Autoencoder. I was looking at various codes online and found most of them in some way or another copy Francois Chollet (Google researchers) code. Now my main ...
1
vote
0answers
24 views

What autoencoder architectures are effective for large images?

I've had easy success in the past making autoencoders for small images using necked down dense networks, but my new application has images of ~1M pixels, which is impractical to address (I think) with ...
0
votes
0answers
129 views

Reparametrization trick for VAE, prooving that the resulting vector follows a normal distribution

So I've been reading about Variational AutoEncoders and I'm stuck on a little exercise meant to help understand the reparameterization trick. Z is a random vector of K elements with a distribution $q(...
0
votes
0answers
22 views

Efficient way to do Autoencoder on large sparse matrix

I have a large csr_matrix of shape (60,000, 180,000) and about 99.7% sparsity. I was trying to train an autoencoder for this matrix via mini-batch optimization. I tried batch size of 6000 with ...
0
votes
1answer
62 views

Unsupervised VAE model? [closed]

I would like to use VAE model in unsupervised learning to generate new feature. Most of the examples are supervised and semi-supervised learning. Where can I find for unsupervised learning or can it ...
0
votes
1answer
44 views

On evaluating variational autoencoders with prior likelihood and reconstruction error

A common evaluation metric for variational autoencoders (VAEs) is estimating the marginal likelihood of some held-out data, i.e. $p(x)$. This is difficult and often one can only get a lower bound. It'...
0
votes
0answers
12 views

faulty autoencoder [duplicate]

I am developing an autoencoder for CIFA10 dataset, without adding noise at the input (which is 2nd goal). The Convnet based autoencoder is not converging: Any suggestions ...
1
vote
0answers
104 views

training prior network on conditional variational autoencoder

The objective function of the conditional VAE is defined as: $L_{CVAE} = -KL(q(z|x,y)||p(z|x)) + \frac{1}{L}\sum_{l=1}^{L}\text{log } p(y|x,z^{(l)})$ Here x is input; y is output; z is latent ...
0
votes
1answer
77 views