How to extract crucial features to create an image

Question

Imagine, you have a dataset containing pictures of (example only, just to explain the task) cats and dogs. The data set is labeled, so we can train using supervised learning algorithms.

My goal is to make a cat from a dog. How to do it? For now, I have a couple of ideas which I can share:

• Use a convolutional autoencoder and train on cats, then give a picture of a dog and see the result (I suppose, it will show the most "similar" cat, so the goal is reached)

• Use an algorithm like GAN to transfer "style". I have no idea whether it is possible or not, but looks like a working idea

Which approaches could you recommend to try out?

Answer 1

I would be more inclined to use something like StyleGAN and play around with the latent space to find the direction that transforms the cat image to dog images and vice versa. I learned a lot from this Video I highly recommend watching it if you have time.

Answer 2

The difference between dog and cat is a little more than just "style". Of course "style" itself is not really well defined, but generally I would think the difference between dog and cat is more content than style.

CycleGAN is a general framework for translating between two domains. It works on unpaired data (that is, for each cat image, you don't have a dog image where the dog is in the exact same location in the exact same pose).