Say I have created some numerical embedding for each word in a text corpora -- that is, for each word I have some n-dimensional vector representing the word. Now say I would like to perform some modelling task using those embeddings on various documents. As such, I'd want to aggregate the word embeddings per document, based on their frequency in the document, to create a single vector representing the document.
A simple way to do this would be to do a weighted average of the word embeddings based on their frequency. However, I might also want to capture the extremes of the vectors by capturing the min and the max (and concating that vector to the weighted average as new features). The min or max word embedding however could just be of extremely low frequency and not really representative of the extremes of the document.
Is there then a way to account for the vectors' weight in coming up with a min or max similar to average? Or restated, is there a way to represent the proportional "extremes" of the vector?
I imagine there must be some arbitrariness involved in such, but I figure there is a principled approach. I've thought perhaps of a weighted avg of upper/lower quantiles but suspect there might be a better method.
