Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [autoencoders]

Feedforward neural networks trained to reconstruct their own input. Usually one of the hidden layers is a "bottleneck", leading to encoder->decoder interpretation.

0
votes
0answers
10 views

Autoencoder for sparse data

Suppose I have a big (1,000x20,000) sparse (95% of elements are zeros) matrix with counts. I want to use autoencoder to encode-decode this matrix. How should I do it? Are there any tricks or ...
0
votes
0answers
7 views

How to calculate precision-recall curve for multiple binary classifier for multiple classes

I am using autoencoder based multiple binary classifier for text regeneration each trained on data related to multiple classes. In other words, each classifier is trained to classify only one class. I ...
1
vote
0answers
10 views

Training Variational Autoencoders in two steps

I started to use Variational Autoencoder in a project, (and I have a hard time determining the weight for the reconstruction loss and KL-loss). I have an idea of training the VAE in two steps.: Train ...
0
votes
0answers
16 views

how to prepare text data for LSTM autoencoder

My main goal is to come up with some topics using LSTM autoencoder. I want to use 20 news_group data set. after reading lots of material and looking at some GitHub project, I am still not clear how ...
2
votes
3answers
97 views

Why is binary cross entropy (or log loss) used in autoencoders for non-binary data

I am working on an autoencoder for non-binary data ranging in [0,1] and while I was exploring existing solutions I noticed that in many people (e.g., the keras ...
2
votes
1answer
43 views

KL Divergence loss in variational autoencoders

I was studying VAEs and came accross the Loss function that consists of KL Divergence. I wanted to intuitively make sense of the KL divergence part of the loss function. It would be great if somebody ...
0
votes
1answer
65 views

How to achieve variational autoencoder (VAE) with unrestricted input?

For a normal VAE an input and a reconstruction with values in the range of $[0, 1]$ are expected. This is necessary since the log loss only makes sense for this range. If the input is not within $[0, ...
1
vote
0answers
14 views

Are VAE used for missing data imputation in multivariate time series? If not, what is used?

Multivariate time series are, to the best of my understanding, one of the few cases where Deep Learning still hasn't had its AlexNet moment. I'm especially interested to the case where most of the ...
0
votes
2answers
46 views

Stacked shallow autoencoders vs. deep autoencoders

In LeCun et. all "Deep Learning", Chapter 14, page 506, I found the following statement: "A common strategy for training a deep autoencoder is to greedily pretrain the deep architecture by training a ...
0
votes
0answers
97 views

rewriting ELBO to highlight the role of priors

I am reading this paper which rewrites ELBO. I am stuck in verifying the mathematics used for doing the rewriting. Essentially, the paper writes the KL term involved in ELBO as follows (equations 13 ...
0
votes
1answer
31 views

advantage of variational autoencoder

I know that VAE is generative model. However it is also used as a dimensionality reduction method. In this case, what are advantages of VAE?? Also I saw that well-applied vae on mnist, and it was ...
1
vote
1answer
32 views

Removing noise with Variational Autoencoders

I have one question that is related to variational autoencoders: can they be used as a denosing algorithm in the same way as standard denosing autoencoders? I generally see people removing the ...
2
votes
1answer
68 views

Where to include Dropout in stacked autoencoder

I'm using Keras to implement a stacked autoencoder, and I think it may be overfitting. I wanted to include dropout, and keep reading about the use of dropout in autoencoders, but I cannot find any ...
2
votes
1answer
45 views

Image feature extraction using an Autoencoder combined with PCA

Background: I have fairly large dataset of biomedical images (around 10,000 images) of 1920x1920 pixels (after cropping parts of black borders out). My task is to extract the 200 most important ...
0
votes
1answer
20 views

How to reconstruct negative acceleration values using a simple autoencoder?

I am trying to reconstruct the acceleration values of a tri-axial accelerometer using a simple autoencoder. As acceleration values are often negative (e.g, -3.4) therefore using a ReLU activation ...
0
votes
1answer
26 views

How to perform signal denoising from corrupted data

In sensors, the data collected is always noisy. I want to denoise the data using machine learning methods. As per my understanding, the output of the trained algorithm should be the clean signal and ...
5
votes
1answer
74 views

Writing PCA as a special kind of auto-encoder

I would like to know if it's possible to view PCA as a particular type of neural network, however, there's one major stumbling block that I haven't yet been able to get past. Define the following "...
1
vote
0answers
18 views

Constraints on low dimensional representations of data

Is there literature discussing introduction of constraints to loss functions in order to specify certain structures on low dimensional representations? If so, how do they compare the efficacy of the ...
0
votes
0answers
26 views

Variational Autoencoder and Covariance Matrix

Why does the encoder from a variational autoencoder map to a vector of means and a vector of standard deviations? Why does it not instead map to a vector of means and a covariance matrix? Is it ...
0
votes
1answer
31 views

Generative model to generate hidden activations coming from a previously trained hidden layer

I need to train a generative model to generate vectors which resemble the activations of a particular hidden layer of a neural network which has been previously trained. In particular, the hidden ...
1
vote
1answer
37 views

What kind of impact do autoencoders have on final model performance when compared to models trained only on supervised data? [closed]

For example, say we have two datasets, a labeled set (I will call it df_labeled) of nrows=200k and an unlabeled dataset (df_unlabeled) of nrows=800k and we want to build a binary classifier. I clearly,...
0
votes
0answers
16 views

What is the best input for denoise autoencoder for sound/audio data?

I am currently trying to build an autoencoder to de-noise audio data. However I have not found any good articles explaining about the input to the autoencoder, i.e. feature vector. As in speech ...
0
votes
1answer
55 views

When does my unsupervised autoencoder start to overfit?

I am working on anomaly detection using an autoencoder neural network with $1$ hidden layer. This is an unsupervised setting, as I do not have previous examples of anomalies. The input data has ...
0
votes
0answers
20 views

What's the difference between random and deterministic encoder in autoencoders?

I read this paper "Wasserstein Auto-Encoders", and they mention deterministic encoder and random encoder but without stating the difference between them. How can we tell the difference?
0
votes
0answers
58 views

Autoencoder as an optimization (search) problem

We all know that machine learning problems can be modeled as an optimization problem where we are searching for the best set of parameter values in the parameter space that optimizes our objective ...
0
votes
1answer
70 views

Help in Understanding Variational Autoencoder Size of Latent Variables

I'm trying to understand further how a variational autoencoder works beyond the conceptual level. However I'm still confused as to what the "vector of mean and variances" can look like and to digest ...
1
vote
0answers
35 views

Equity Risk Model using an autoencoder

I am trying to create a statistical equity risk model using an autoencoder in a similar fashion to how one would use PCA to derive the systematic and specific risk components of a stock's returns. I ...
0
votes
1answer
50 views

Minimum Input Dimension for Autoencoder Neural Network

Model: Assume we want to learn patterns using an autoencoder neural network. In the simplest case, such a network is "shallow" with 1 hidden layer, takes a $d$-dimensional numerical input vector $x$, ...
0
votes
0answers
31 views

Correct numerical feature transformation for neural networks

Model: I am working on a "shallow" (3-layer) auto encoder neural network. The input layer receives a, say 25-dimensional, vector $x$ of numerical elements representing client purchases. Several ...
1
vote
1answer
30 views

Prerequisites for Wasserstein GAN/Autoencoder

Can someone who read WGAN/WAE papers and understood Wasserstein part, could you share how you prepared necessary Optimal Transport background? The mentioned papers seem little tough if you don't have ...
0
votes
0answers
17 views

What are the shortcomings of calculating the loss in pixel space vs. feature space

While training (Variational)-Autoencoder networks, I came along the paper by Higgins et al. "DARLA" where she stated: The shortcomings of calculating the log-likelihood term [...] on a per-pixel ...
2
votes
1answer
70 views

Generative autoencoders - how important is agreement of latent variable distribution e.g. with Gaussian?

Autoencoders want to minimize distortion of encoding-decoding process, preferably alongside evaluation by discriminant. Generative autoencoders additionally would like latent variable from a chosen ...
1
vote
1answer
59 views

DeepLearning & Anomaly Detection - Understanding & How to Properly Tune

I'm looking into understanding the Deeplearning anomaly detection algorithm provided by h2o. I tried to recreate an example below. Perhaps some of these questions are basic, but I'm trying to better ...
0
votes
0answers
34 views

Categorical data & Gaussian latent variables

I am learning about imposing structure on the latent variables in autoencoders. In that context I have looked at variational autoencoders (VAEs) and adversarial autoencoders (AAEs). This paper ...
0
votes
1answer
106 views

VAE with mixture of gaussian prior

I try to understand this paper where they try to use a mixture of Gaussian as a prior, instead of the standard gaussian. There are several things unclear to me though: They say that they set $\pi_k = ...
1
vote
1answer
53 views

Understand VAE with VamPrior

I am currently reading this paper. The authors propose to use as an prior this expression: $$ p_\lambda(z) = \frac{1}{K} \sum^K_{k=1} q_\phi (z\mid u_k) $$ where $q_\phi$ is the encoder, and $u_k$ is ...
1
vote
0answers
17 views

Use internal representation of autoencoder for anomaly detection [closed]

I've trained an autoencoder to recognize 'positive' time series (the network is a simple fully connected network, no recurrent layers). The problem is that from what my advisor says, I should try to ...
1
vote
0answers
46 views

Forming Conditional Probabilities in tensorflow

This is a repost from stack, but I think it may be more applicable here. Granted there is even a bounty for it there, if someone could answer it. https://stackoverflow.com/questions/53421179/forming-...
2
votes
1answer
40 views

KL-Diverence of Q(z|X) and P(z) in Variational Autoencoder (VAE)

I aim to understand how $D_{KL}[Q(z | X) || P(z)]$ can be converted to $\frac{1}{2} \sum_{k} (\Sigma(X) + \mu^{2}(X) - 1 - \log \Sigma(X))$, where $k$ is the dimension of the Gaussian distribution. ...
0
votes
1answer
55 views

Implementation of WAE-GAN does not match with the description in the paper

According to the litterature and specifically to this paper, the wasserstein autoencoders is an encoder-decoder architecture. So it must contain encoder and decoder parts. in the algorithm ...
3
votes
2answers
103 views

Encoder Decoder networks with varying image sizes

Encoder Decoder Network - Computerphile : At the very beginning of this video, Michael Pound goes on to say: So it (encoder decoder network) makes no assumptions about the size of the input the ...
0
votes
0answers
59 views

Can attention be implemented without encoder / decoder?

I just got into models beyond biLSTM, would like to start with applying attention to my existing network (RNN). I find examples for attention always with encoder decoder architecture, however is it ...
1
vote
2answers
254 views

convolutional autoencoder on an odd size image

I am trying to apply convolutional autoencdeor on a odd size image. Below is the code: ...
1
vote
1answer
24 views

How can I tell if an Autoencoder is encoding my data properly?

Autoencoders can be classified as a method of unsupervised learning, and unsupervised learning often comes with a problem where it's hard to tell if the model is working properly. However, some ...
1
vote
1answer
58 views

Using step function as activation function in the final layer

I am using variational autoencoders as machine learning algorithm. My input data are images/matrices that represent user interface layouts or how the HTML page will be divided. I am thinking to ...
0
votes
0answers
155 views

Autoencoder with small dataset - simple images

I wanted to ask a simple question regarding autoencoders to parse for tips and possible advice before diving into this path. I have a small dataset ~50-100 heatmap images that are relatively simple. ...
0
votes
1answer
50 views

back propagation in neurons with zero weight and some specific conditions

I have read a lot of articles to understand what is happening behind the scene in backpropagation like Ive gone through this and many other like that. I think I understand how the backpropagation ...
1
vote
0answers
82 views

How to evaluate performance of (variational) autoencoders?

Let's assume that you have trained your (variational) autoencoder on MNIST digits. After some time, you check the result and decide that the reconstruction is pretty good. But this is highly ...
0
votes
0answers
10 views

How to assign labels to un-labeled documents

I have a bunch of text documents and they are not labeled but each document represents one or more than one category/label. I'd like to assign the appropriate label (s) to each document. What's the ...
0
votes
1answer
174 views

Discrete Random Variables and Deep Generative Models - Why Gumbel-Softmax is needed?

I am reading this 2014 NIPS paper on deep generative models and their application to latent discrete random variables, and this 2017 ICLR paper on Gumbel-Softmax. I essentially don't understand why we ...