What is embedding? (in the context of dimensionality reduction)

Question

In the context of dimensionality reduction one often uses word embedding, which seems to me a rather technical mathematical term, which rather stands out compared to the rest of the discussion, which in case of PCA, MDS and similar methods is just the basic linear algebra.

Yet, I would rather avoid using/interpreting this term too loosely. So, what embedding really is: the low-dimensional subspace hidden within a bigger one? The projections of the data vectors onto this subspace? The projection operator mapping the higher-dimensional space onto the lower-dimensional one, as suggested here and here? Something else?

Thank you for clarifications and examples.

Answer 1

The word is used ambiguously, e.g. this quote from Google crash course on machine learning says [my comments in square brackets]:

An embedding is a relatively low-dimensional space [subspace] into which you can translate high-dimensional vectors. Embeddings make it easier to do machine learning on large inputs like sparse vectors representing words. Ideally, an embedding captures some of the semantics of the input by placing semantically similar inputs close together in the embedding space [projection]. An embedding can be learned and reused across models [mapping].

Depending on context, it's best that you make it precise if you are talking about embedding layer in neural network (the function), or the generated embeddings matrix, or embeddings in short (the projections), or you could directly say that you are taking about embedding space. Usually the word embedding would be used to describe the projection of a word into the latent vector, e.g. "embeddings were created using BERT model".