I have n vectors with m features and also a weight vector with n elements. I'd like to reduce the number of n vectors in a way that the probability distributions of the m (weighted) features (across the n vectors) are maintained (or "least" affected).
2 Vectors can be collapsed into 1 by weight averaging their m features with each of the vector's weights (the resulting weight is the sum of the two vector's weights).
m is in the range of 2..5, n can be up to 500. The distributions of the features are unknown and different for each feature.
I would require a distance measure for two probability distributions (not sure if some combination of variance and mean is sufficient) and a smart way of selecting the two candidate vectors to merge.
At the moment I am using a Euclidian distance measure and merge the smallest weight's vector with the closest (wrt. distance measure) other vector. There is no check of whether or not the distributions are maintained (just followed intuition).
Are there any established methods to do what I describe or any keywords I could follow up on? Any help or comments are appreciated.
I may have found ways to measure the distance between two probability distributions (the original distribution and the distribution from reducing the data points).
I am unsure of which one to choose though.