I am working on a clustering project on a dataset that has some numerical variables, and one categorical variable with very high cardinality (~200 values). I was thinking if it is possible to create an embedding for that feature exclusively, after one-hot encoding (ohe) it. I was initially thinking of running an autoencoder on the 200 dummy features that result from the ohe, but then I thought that it may not make sense as they are all uncorrelated (mutually exclusive). What do you think about this?

On the same line, I think that applying PCA is likely wrong. What would you suggest to find a latent representation of that variable?

One other idea was: I may use the 200 dummy ohe columns to train a neural network for some downstream classification task, including an embedding layer, and then use that layer as low-dimensional representation... does it make any sense?

Thank you in advance!

  • $\begingroup$ What is the purpose of obtaining the latent representation? $\endgroup$
    – Sycorax
    Jan 31, 2023 at 21:01
  • $\begingroup$ You can use tf.keras.layers.Embedding: keras.io/api/layers/core_layers/embedding $\endgroup$
    – Amin Shn
    Jan 31, 2023 at 22:08
  • $\begingroup$ @Sycorax I want to run a clustering algorithm, so I need to reduce the dimensionality of that ohe'ed categorical variable. $\endgroup$ Feb 1, 2023 at 8:31
  • $\begingroup$ @AminShn you mean using the embedding layer in a surrogate classification model? $\endgroup$ Feb 1, 2023 at 8:31
  • $\begingroup$ You asked how to embed a categorical variable without ohat, and I said that's possible using the Embedding layer which maps each category to a higher dimensional vector in the latent space. $\endgroup$
    – Amin Shn
    Feb 1, 2023 at 8:41