# “Importance” metric for discrete variable value

(I am far from an expert in the field of statistics, so I apologize beforehand if my question is irrelevant or my use of any terms is incorrect)

Let's suppose that we have a discrete variable $X$, and a set of $n+1$ observations $\{x_0, x_1, \dots, x_n\}$. The values of $X$ may not be numeric and all observations are considered equal with no particular order. Therefore, we can transform the observation set above to a set of $k+1$ value/frequency pairs $\{<v_0, f_0>, <v_1, f_1>,\dots,<v_k, f_k>\}$.

Is there a metric that estimates the "importance" of a specific value in this set?

For example, in an observation set where all values exist exactly three times, none can be considered more important than the others. On the other hand, in a set where all but one values exist 3 times, I would want that one value that exists 10 times to be differentiated from the rest. Same with a value that only appears once.

A simple probability calculation, for example, $P(k) = f_k / (n+1)$ would not be of help, because it does not take into account the other values.

I experimented with various combinations/formulas using the various layman-known metrics (mean, standard deviation, maximum, minimum etc), but anything I came up with seemed too arbitrary for me to trust.

I am currently using the standard score of the value frequency as an estimator, but it has a major issue: it's not bounded, and I am not at all sure how to normalize it without unknowingly biasing any results.

I would appreciate any reference to a metric that might get me started, or to any terminology that describes what I'd like to do more accurately, so that I can focus my search.

I would especially appreciate any bounded metrics that can be computed incrementally as new samples are added to the observation set :-)

-
I think we may need more information, in particular, what is this metric to be used for? What are the particular characateristics that make a value "important"? For example, what if one value occurs 3 times, and all the other 10 times? What if half occur 3 times and half occur 10? –  Simon Byrne Jun 30 '11 at 11:40
@Simon Byrne: The metric would be used to provide (part of) a weight coefficient in a similarity calculation. Conceptually, I'd like to penalise values that are dominant within a population, on the premise that they are not as distinguishing as less frequent ones. The only characteristic I have to work with is the frequency of a value - the value itself does not matter. –  thkala Jun 30 '11 at 12:19
@Simon Byrne: I suppose that a bounded (e.g. within [-1, 1]) metric that could tell me how much does a specific observation deviate from the rest of the population would also do - I could apply it on the value frequency set. –  thkala Jun 30 '11 at 12:28
@thkala: Are you trying to match records based on the similarity of different variables? If so, there is quite a lot of literature on this, usually under the names of "probabilistic matching" or "probabilistic record linkage" –  Simon Byrne Jun 30 '11 at 12:29
@thkala Research "entropy" while you're at it. (Entropy can be interpreted as an average "importance" where importance is measured as the negative logarithm of the frequency.) –  whuber Jun 30 '11 at 13:52