Questions tagged [autoencoders]

Feedforward neural networks trained to reconstruct their own input. Usually one of the hidden layers is a "bottleneck", leading to encoder->decoder interpretation.

Filter by
Sorted by
Tagged with
2
votes
0answers
59 views

Attention with auto-encoders for feature extraction

I understand the use of attention mechanisms in the encoder-decoder for sequence-to-sequence problem such as a language translator. I am just trying to figure out whether it is possible to use ...
0
votes
0answers
25 views

How do they use their dataset with VAEs?

Old Photo Restoration via Deep Latent Space Translation (https://paperswithcode.com/paper/old-photo-restoration-via-deep-latent-space) In the article, it says : "We propose to restore old photos ...
1
vote
0answers
24 views

Probabilistic Difference between Autoencoders and Variational Autoencoders

I have recently read up about Autoencoders and Variational Autoencoders. In Variational Autoencoders, the loss is modeled based on what distribution we choose for P(x|z). So, if we choose it to be ...
2
votes
0answers
13 views

Why are the tied weights in autoencoders transposed and not inverted?

I am currently reading about Autoencoders. From what I understand so far, when we are dealing with a symmetrical autoencoder, a good practice is to tie the weights of the decoder layers to the weights ...
1
vote
0answers
25 views

Why KL Divergence instead of Cross-entropy in VAE

I understand how KL divergence provides us with a measure of how one probability distribution is different from a second, reference probability distribution. But why is it particularly used (instead ...
0
votes
1answer
36 views

Why VAE need reparameterization trick while LDA does not (both using variational inference for optimization)

Both VAE and LDA (latent Dirichlet allocation) is based on variational inference, and they both try to optimize ELBO objective function Variational autoencoders use reparameterization so that "...
0
votes
0answers
7 views

Understanding Training Word Embeddings?

I am new to Natural Language Processing. I am trying to understand how word embeddings are created. When we are training Neural Networks, it is usuallly the weights of the neural network that are ...
0
votes
0answers
9 views

DL model for solving a multi-class classification problem works fine for the first class and then the performance gradually drops for other classes

I designed an AE shaped deep neural network to perform a multi-label class classification. The classes are not mutually exclusive. The last layer has n neurons; one responsible for each class. I pass ...
0
votes
0answers
14 views

Should convolutions or transposed convolutions be used in the decoder part of a Conv-based autoencoder?

I am implementing a convolutional autoencoder. For the decoder part of the model, some examples (such as this one from Francois Chollet) use standard convolutional layers (Conv2D in keras) in the ...
0
votes
0answers
11 views

Which metrics should be used in preprocessing in continual learning?

So my idea is to train an LSTM - autoencoder for anomaly detection by continual learning, i.e., I want to update the model after each 10 time steps. Firstly I will train it on source data, then re-...
0
votes
0answers
25 views

Variational Autoencoder with a Flexible Prior

Let's say I have a Variational autoencoder with a Gaussian prior architecture and I want to regularize this VAE with a flexible prior. Does sampling with the normal "reparameterization trick"...
1
vote
0answers
13 views

Implementation of factorVAE, understanding distributions

I'm trying to understand and implement the factor-VAE and I'm using the paper Disentangling by Factorising. My probability background is weak and therefore I have issues understanding basic concepts, ...
1
vote
0answers
12 views

autoencoders for radiographs - do watermarks affect the performance considerably

I have to implement an autoencoder to reconstruct the input radiographs and do unsupervised feature learning in the process. However, the radiographs that I have contain some watermarks like X-ray ...
0
votes
0answers
5 views

Layers architecture in an autoencoders vs input shape

I am working with a highly imbalanced data for a multi-class classification task. I choose to adopt an approach I read from a paper where they used an ensemble of autoencoders to develop the model, ...
2
votes
2answers
68 views

Should reconstruction loss be computed as sum or average over input for variational autoencoders?

I am following this variational autoencoder tutorial: https://keras.io/examples/generative/vae/. I have included the loss computation part of the code below. I know VAE's loss function consists of the ...
0
votes
0answers
13 views

Variational Encoder on typical PCA task

I have just started to learn about variational autoencoders. As a first step I have tried doing a task with both PCA and VA. However, the VA results are very poor. Does anybody see any quick fixes ...
0
votes
0answers
19 views

Autoencoder Incorrect Output/Predict - Model Built and Trained. *Please Assist*

Background Let me preface, I am new to python and machine learning. I have been tasked with creating an autoencoder to reduce dimensionality on a made-up dataset (proof of concept). I am working in ...
0
votes
0answers
14 views

Is a differentiable quantile function a sufficient condition for performing the reparametrization trick with respect to a distribution?

The goal with the "reparametrization trick" is to to sample a noise variable $\epsilon$ from a simple distribution of our choice (like a standard Gaussian). We want to apply a deterministic ...
1
vote
1answer
39 views

Variational Autoencoder (VAE) latent features

I'm new to DL and I'm working on VAE for biomedical images. I need to extract relevant features from ct scan. So I created first an autoencoder and after a VAE. My doubt is that I don't know from ...
0
votes
0answers
15 views

Latent space distribution in conditional VAE

I have a simple conditional variational autoencoder which takes two one-dimensional inputs x and y, where y = x +z, where x~N(0,1) and z~(mu,std). The latent space is also one-dimensional (mu, log_var)...
0
votes
1answer
27 views

How to use transfer learning for autoencoder based anomaly detection?

I have 2 data sets which are somehow similar and I want to use them for domain adaptation. Dataset1 is imbalanced and consists of labeled positive and negative samples. Dataset2 consists of only ...
0
votes
0answers
10 views

How to construct transfer learning based autoencoder model?

I want to train an autoencoder for anomaly detection (train on normal samples, compute reconstruction error and classify as anomalies all new samples with "too high" reconstruction error). ...
0
votes
1answer
28 views

Autoencoder based anomaly detection: how to train AE also with outliers?

Suppose the data without labels, i.e., unsupervised anomaly detection task. The data are multivariate sequences, so the idea is to use LSTM based autoencoder (AE). However, typically AE-s for anomaly ...
2
votes
0answers
12 views

Why do tied weights in Autoencoders *force* divergence in the features?

Imagine the following scenario: Given a signal x, you pass it through a bank of convolutional filters to get a feature map z (...
0
votes
0answers
8 views

Find autoencoder best architecture for anomaly detection

Suppose we have N variables, and we construct a standard autoencoder MLP with N input nodes, N output nodes, and one hidden layer with M nodes. Question is: is there an algorithm to find the best M to ...
1
vote
0answers
23 views

Deep Autoencoder for Narrow Dataset Feature Extraction

#Background# Hello, just to preface my question, I am relatively new to python and to machine/deep learning in general. Part of my job is now to implement an Autoencoder that reduces the ...
0
votes
1answer
32 views

Auto-encoders' learning process and overfitting

Reading about autoencoders from Ian Goodfellow's deep learning book, and they made this statement about autoencoders learning process on page 494: "Unfortunately, if the encoder and the decoder ...
1
vote
0answers
37 views

Can binary cross entropy loss be used for non-binary data? [duplicate]

I am following this keras tutorial to construct a convolutional MNIST autoencoder. The decoder has a sigmoid activation function and the entire autoencoder is trained with binary_crossentropy loss. ...
0
votes
0answers
64 views

Differentiable PCA?

Is there a differentiable method for dimensionality reduction that is either based on PCA or has the properties of: Mathematically or algorithmically defined, e.g. not trained like an ML model or t-...
2
votes
2answers
57 views

Intuitive Explanation of “AutoEncoders”

To my knowledge, Autoencoders are an unsupervised learning technique in which we leverage neural networks for the task of representation learning. I know there is a lot of topics around autoencoders, ...
0
votes
0answers
20 views

Creating CNN-Autoencoder From CNN With Non-Square Kernels

I'm attempting to create a CNN-Autoencoder model for an unsupervised clustering problem on adjacency matrix data. The CNN I'm working with, BrainNetCNN, is unconventional in that its filters aren't ...
0
votes
0answers
8 views

Sequence sub matching for a big database

I have database (~1M) of 1D sequence (variable lengths). I want to be able to match a query sequence against the DB in real time. The query may be a sub sequence of one (or multiple) of the database ...
0
votes
0answers
8 views

Latent Dimension For AutoEncoder network

I have been experimenting with auto-encoder network to build content based image search engine. While i have had early success, one of the question which I am still not been able to figure out is &...
0
votes
2answers
69 views

How to benchmark performance of an autoencoder

An Autoencoder is defined as a device that can extract useful features from data, and also use those features to reconstruct initial data. I'm trying to understand what the word "useful" ...
0
votes
1answer
27 views

Confused by the latent variables in normalizing flow theory

I'm looking through the paper on variationl inference in normalizing flow and have difficulties with understanding some ideas. I know there are latent variables $\mathbf{z}_i$ and observed variables $\...
0
votes
1answer
34 views

Understanding reparameterization trick and training process in variational autoencoders

I am trying to understand variational autoencoders, particularly the sampling component and the reparameterization trick. I understand that instead of using a fixed determinstic latent representation ...
0
votes
0answers
12 views

How to validate an unsupervised method that uses doc2vec and autoencoder

Background : I have an array of paragraphs, where each paragraph is a description for a movie Objective : I am trying to find a novelty value for each paper compared to the whole corpus Method : I ...
2
votes
1answer
28 views

what is -0.5 in VAE loss function with KL term

The VAE loss is composed of two terms: Reconstruction loss KLD loss in the implementation there is -0.5 applied to KLD loss. Kindly let me know what is this -0.5
5
votes
1answer
59 views

When was Auto-Encoder used for anomaly detection for the first time?

I would like to cite Auto-Encoder based solution for anomaly detection, however I can't find the origin. When was Auto-Encoder used for anomaly detection for the first time?
0
votes
0answers
10 views

RSGAN - how do you separate parts of an image using a VAE

Authors of this paper (RSGAN) mention they're able to use two autoencoders, one separating face from an image and the other one separating hair: They don't describe how this is being done and it isn'...
3
votes
0answers
53 views

How do you calculate log likelihood p(x) for a VAE?

I was reading the Importance Weighted Autoencoders paper and its experiment section compares the density estimation result on MNIST for IWAE vs VAE. I know that density estimation estimating log p(x) ...
2
votes
1answer
125 views

Reparametrization trick

I am thinking about the reparameterization trick in a variational autoencoder. I know that it can be used with normal distribution. Can the reparameterization trick be applied to other distributions ...
1
vote
0answers
20 views

Gradient boosting and feature to target correlations?

I'm working on a data science challenge, I have a tabular data with around 80 features (79 predictors and a target variable between 0 and 1). I've tried two approaches: Using autoencoders to do ...
6
votes
4answers
82 views

What are some non-toy applications of autoencoders?

I haven't come across a real world application of autoencoders before. Usually, for dimensionality reduction I've used PCA or random projections instead. Most examples I've come across of using ...
2
votes
0answers
30 views

Identify the parameter causing the anomaly in a multivariate dataset

I have a payment transaction dataset with a large number of predictor variables. I am trying to build a model for anomaly detection and I have evaluated various algorithms/approaches for the same like ...
2
votes
1answer
41 views

understanding 0 kl-divergence loss in beta variational autoencoders

I've been reading this paper "Understanding disentangling in beta-VAE" and there was one part I was confused about. https://arxiv.org/abs/1804.03599 (paper link) On Figure 4, it shows that if the KL-...
2
votes
2answers
98 views

PCA vs linear Autoencoder: features independence

Principal component analysis is a technique that extract the best orthogonal subspace in which we can project our points with less information loss, maximizing the variance. A linear auto encoder is ...
3
votes
1answer
186 views

PCA with polynomial kernel vs single layer autoencoder?

What is the relationship between PCA with polynomial kernel and a single layer autoencoder ? What if it is a deep autoencoder?
0
votes
0answers
13 views

Denoising autoencoder with oversampling?

Denoising autoencoder is using noised added training samples to predict (original) training samples themselves. The goal is to denoise when being applied to the real sample. Here is an example of ...
0
votes
0answers
14 views

Force Autoencoder to Learn Certain Features

I am given a set of small 32x32x1 images that each contain a single object. I want to extract certain features from these images, and I know apriori which features I want. For example, I know that I ...

1
2 3 4 5
10