I learned to use autoencoder and variational autoencoder for images, and I would like to apply these techniques to other types of non-image data, such as common tabular data we can find in Kaggle contest, for denoising and data generation.

However, now I have a problem with a general dataset for this kind of generative model: unlike image data, each feature has a quite different distribution.

  • For image data, we have many pixels, and we can treat them as equivalent by having the same form of loss function for each pixel.
  • On the other hand, there is no such feature-to-feature equivalence for a general dataset, and each feature can have a quite distinct distribution, even after standardization of each feature. For example, the data can have a Gaussian distributed feature 1, and a Poisson distributed feature 2 shifted by feature normalization, and a Cauchy distributed feature 3.

In that case, I guess we should not directly use the squared error for each feature as the loss function, as we should not expect a Gaussian noise for each feature with equal variance. Then what is an appropriate procedure to obtain the loss function for a general dataset?

I feel this question is related to the metric of distance in high-dimensional space, as I am having the same concern for using kernels or calculating distances for these nonequivalent features.

  • $\begingroup$ Did you consider using embeddings for categories, like in this paper? $\endgroup$ – Jakub Bartczuk Mar 12 '18 at 11:45
  • $\begingroup$ @JakubBartczuk: I'm sorry but I fail to see the connection between this paper and my question. $\endgroup$ – DiveIntoML Mar 12 '18 at 14:51
  • $\begingroup$ You mentioned tabular data. Tabular data often contains categorical features $\endgroup$ – Jakub Bartczuk Mar 12 '18 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.