# Data reduction by maintaining data distribution

I have n vectors with m features and also a weight vector with n elements. I'd like to reduce the number of n vectors in a way that the probability distributions of the m (weighted) features (across the n vectors) are maintained (or "least" affected).

2 Vectors can be collapsed into 1 by weight averaging their m features with each of the vector's weights (the resulting weight is the sum of the two vector's weights).

m is in the range of 2..5, n can be up to 500. The distributions of the features are unknown and different for each feature.

I would require a distance measure for two probability distributions (not sure if some combination of variance and mean is sufficient) and a smart way of selecting the two candidate vectors to merge.

At the moment I am using a Euclidian distance measure and merge the smallest weight's vector with the closest (wrt. distance measure) other vector. There is no check of whether or not the distributions are maintained (just followed intuition).

Are there any established methods to do what I describe or any keywords I could follow up on? Any help or comments are appreciated.

Update

I may have found ways to measure the distance between two probability distributions (the original distribution and the distribution from reducing the data points).

I am unsure of which one to choose though.

• What's wrong with weighted random sampling, besides merging similar objects? Also 500 is tiny. Apr 28 '16 at 7:04
• All objects are used in the end. Sampling doesn't work as it would potentially sample the same object twice (which wouldn't be allowed). Essentially the weighted average of m and the sum of weights has to be preserved by whatever data reduction transformation is applied. Apr 28 '16 at 8:24
• I'm glad to hear that 500 is tiny ;-) Apr 28 '16 at 8:25
• Well, you could sample without replacement (but that still won't have your other criterions). Try to be explicit about your requirements. Preserving the global weighted averages exactly is a restriction. Also, apparently your vectors are probability distributions, i.e. every vector has a sum of 1? Apr 28 '16 at 9:14
• Across all vectors n, for each feature dimension in m you can plot a distribution in a histogram, but the values don't sum to 1 (neither do the n weights for each vector). Apr 28 '16 at 9:48

What do you want to optimize? You want to merge objects by taking the weighted average, i.e. $z_i=(w_{xi}\cdot x_i+w_{yi}\cdot y_i)/(w_{xi}+w_{yi})$; and $w_{zi}=w_{xi}+w_{yi}$ I assume? Because that will preserve the global averages and weight sum (see comments on the question).