I learned to use autoencoder and variational autoencoder for images, and I would like to apply these techniques to other types of non-image data, such as common tabular data we can find in Kaggle contest, for denoising and data generation.
However, now I have a problem with a general dataset for this kind of generative model: unlike image data, each feature has a quite different distribution.
- For image data, we have many pixels, and we can treat them as equivalent by having the same form of loss function for each pixel.
- On the other hand, there is no such feature-to-feature equivalence for a general dataset, and each feature can have a quite distinct distribution, even after standardization of each feature. For example, the data can have a Gaussian distributed feature 1, and a Poisson distributed feature 2 shifted by feature normalization, and a Cauchy distributed feature 3.
In that case, I guess we should not directly use the squared error for each feature as the loss function, as we should not expect a Gaussian noise for each feature with equal variance. Then what is an appropriate procedure to obtain the loss function for a general dataset?
I feel this question is related to the metric of distance in high-dimensional space, as I am having the same concern for using kernels or calculating distances for these nonequivalent features.