# What are the shortcomings of calculating the loss in pixel space vs. feature space

While training (Variational)-Autoencoder networks, I came along the paper by Higgins et al. "DARLA" where she stated:

The shortcomings of calculating the log-likelihood term [...] on a per-pixel basis are known and have been addressed in the past by calculating the reconstruction cost in an abstract, high-level feature space given by another neural network model, such as a GAN [..] or a pre-trained AlexNet [...]

Since there is no statement regarding why it is better to use feature-space instead of pixel-space, I am wondering why the results become better in the first case. I assume that since the per-pixel loss encourages the whole image to be trained correctly, a feature based loss can only fetch the perceptual similarity. Therefore, the per-pxel loss should always result in better outputs, or is there any important influence by the regularizer which I am missing?

• @Tik0 Well... it still optimizes an ELBO, although not the same one. The generative likelihood model $\log p(x|z)$ has simply changed. For references, I don't know of a specific one. Maybe Dosovitskiy and Brox Generating Images with Perceptual Similarity Metrics based on Deep Networks is relevant. – user3658307 Apr 4 '19 at 12:53