Latent space refers to an abstract multi-dimensional space containing feature values that we cannot interpret directly, but which encodes a meaningful internal representation of externally observed events.
Just as we, humans, have an understanding of a broad range of topics and the events belonging to those topics, latent space aims to provide a similar understanding to a computer through a quantitative spatial representation/modeling.
The motivation to learn a latent space (set of hidden topics/ internal representations) over the observed data (set of events) is that large differences in observed space/events could be due to small variations in latent space (for the same topic). Hence, learning a latent space would help the model make better sense of observed data than from observed data itself, which is a very large space to learn from.
Some examples of latent space are:
1) Word Embedding Space - consisting of word vectors where words similar in meaning have vectors that lie close to each other in space (as measured by cosine-similarity or euclidean-distance) and words that are unrelated lie far apart (Tensorflow's Embedding Projector provides a good visualization of word embedding spaces).
2) Image Feature Space - CNNs in the final layers encode higher-level features in the input image that allows it to effectively detect, for example, the presence of a cat in the input image under varying lighting conditions, which is a difficult task in the raw pixel space.
3) Topic Modeling methods such as LDA, PLSA use statistical approaches to obtain a latent set of topics from an observed set of documents and word distribution. (PyLDAvis provides a good visualization of topic models)
4) VAEs & GANs aim to obtain a latent space/distribution that closely approximates the real latent space/distribution of the observed data.
In all the above examples, we quantitatively represent the complex observation space with a (relatively simple) multi-dimensional latent space that approximates the real latent space of the observed data.
The terms "high dimensional" and "low dimensional" help us define how specific or how general the kinds of features we want our latent space to learn and represent. High dimensional latent space is sensitive to more specific features of the input data and can sometimes lead to overfitting when there isn't sufficient training data. Low dimensional latent space aims to capture the most important features/aspects required to learn and represent the input data (a good example is a low-dimensional bottleneck layer in VAEs).
If this answer helped, please don't forget to up-vote it :)