How do they use their dataset with VAEs?

Question

Old Photo Restoration via Deep Latent Space Translation (https://paperswithcode.com/paper/old-photo-restoration-via-deep-latent-space)

In the article, it says : "We propose to restore old photos that suffer from severe degradation through a deep learning approach. Unlike conventional restoration tasks that can be solved through supervised learning, the degradation in real photos is complex and the domain gap between synthetic images and real old photos makes the network fail to generalize ... "

I have a bit of difficulties to understand how they built their dataset of photos. At the beginning, I thought the idea was to create synthetic old photos from a set modern photos like IMDB-WIKI – 500k+ face images with age and gender labels. Hence, I thought the idea was to use deep learning with the synthetic photos dataset (artificially degraded) as input and the modern photos as target. Once the model is train, we could've use it with real old photos instead of artificially build photos. It seems not the way to go. Can you explain to me how they generate their dataset and use it in their VAEs?

Answer 1

This is exactly the way to go, the way I (and some other colleagues) read it. They detail the data generation process in section 4.2 (Data generation). The VAE1 is only there to get the most important features of the photos into a latent space that is then translated into a different latent (discovered with VAE2) space via Mapping T. There is a twist on the VAE1, though, as they use a GAN to align the domains of the latent spaces. They use the Pascal VOC 2012 Dataset if I recall.