3
$\begingroup$

In the context of dimensionality reduction one often uses word embedding, which seems to me a rather technical mathematical term, which rather stands out compared to the rest of the discussion, which in case of PCA, MDS and similar methods is just the basic linear algebra.

Yet, I would rather avoid using/interpreting this term too loosely. So, what embedding really is: the low-dimensional subspace hidden within a bigger one? The projections of the data vectors onto this subspace? The projection operator mapping the higher-dimensional space onto the lower-dimensional one, as suggested here and here? Something else?

Thank you for clarifications and examples.

$\endgroup$
  • $\begingroup$ @Tim Thank you. In principle, I have now my question answered. However my formulation is different from those given elsewhere (is embedding a subspace OR a projection OR and operation/function), so you might consider reopening it for the benefit of the community. $\endgroup$ – Vadim Sep 15 at 8:58
2
$\begingroup$

The word is used ambiguously, e.g. this quote from Google crash course on machine learning says [my comments in square brackets]:

An embedding is a relatively low-dimensional space [subspace] into which you can translate high-dimensional vectors. Embeddings make it easier to do machine learning on large inputs like sparse vectors representing words. Ideally, an embedding captures some of the semantics of the input by placing semantically similar inputs close together in the embedding space [projection]. An embedding can be learned and reused across models [mapping].

Depending on context, it's best that you make it precise if you are talking about embedding layer in neural network (the function), or the generated embeddings matrix, or embeddings in short (the projections), or you could directly say that you are taking about embedding space. Usually the word embedding would be used to describe the projection of a word into the latent vector, e.g. "embeddings were created using BERT model".

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.