# Weighted entropy as a measure of diversity

Suppose that you are a company manager and you are looking for a statistical measure that defines the international reputation of your company. So, you collect data on your clients and the countries they come from.

You want this measure to have the following properties:

1. The larger the number of countries where your client come from, the more your company is international.

2. If all of your clients are from a single country, i.e., the number of countries is 1, your international reputation should be the lowest value (let's say 0 for convenience).

3. Unfortunately, you have to discriminate between the countries on your client list. For business reasons, you prefer to have more clients from high-income countries and fewer clients from low-income countries and there are countries that you can't deal with because of sanctions, embargoes or similar reasons.

Let's say you have the following data:

1. $n_i$: The number of clients from the i-th country.

2. $p_i$: The fraction of your clients that come from the i-th country.

3. $w_i$: The desirability of dealing with clients from the i-th country.

4. $N$: The sum of $n_i$'s. i.e. the number of countries.

I am currently reading about Rényi entropy as a potential method. Rényi entropy of order $q$ is given by the formula

$$^qH = \frac{1}{1-q} \ln\left(\sum_{i=1}^{N}p_i^q\right)$$

And I'm thinking of modifying it to a new formula taking "desirability coefficients" $w_i$ into account. My suggestion is this:

$$^qH = \frac{1}{1-q} \ln\left(\sum_{i=1}^{N}w_ip_i^q\right)$$

Do you think this is a valid method? Are there any better methods or suggestions available?

UPDATE:

One way to take the desirability of trade into account is to update the probabilities in this way:

$$\tilde p_i = \frac{w_i\cdot n_i}{\sum_{i=1}^N(w_i \cdot n_i)}$$

But I'm not sure if this actually works. How can I know that it's a good choice?

• For weighting probabilities there's much more choices (often argued as better) than weighted mean, e.g. weighted geometric mean: stats.stackexchange.com/questions/155817/…
– Tim
Commented Feb 29, 2016 at 12:19

You want some effective formula, so there is a lot of freedom you can do.

For rescaling I would do something slightly different:

$$\tilde{p}_i = \frac{w_i p_i}{\sum_i w_i p_i}$$

where $\tilde{p}_i$ is the estimated percent of money coming from country $i$. So that if e.g. Germans pay twice as much as Poles, you weight them twice as much.

Then you can use $H_q(\{\tilde{p}_i\})$.

BTW: For communicating results I would rather use some diversity index (see https://stats.stackexchange.com/a/135153/6552 or https://stats.stackexchange.com/a/144235/6552). Then you can say something like "we have money coming from effectively 3.4 countries".

• Do you suggest to use $^q H= \frac{1}{1-q} \ln ( \sum_{i=1}^N (\frac{w_i p_i}{w})^q)$ where $w=\sum_i w_i$? In other words, you're adjusting the probabilities in a way that the coefficients determine the new probabilities. Is that right? But the new probabilities do not sum to 1. Would you please extend your answer and explain why you chose that? Commented Jan 21, 2016 at 8:24
• OK. I thought about it again. The approach you're suggesting is different. What I had in mind is to adjust the probabilities by multiplying the weights and the number of customers from the i-th country, and then divide by the sum of all such multiplications for all i's. That's a different approach, but I guess that one would work too. Please see the update on my main post. Commented Jan 21, 2016 at 8:47
• @H.Z. My error, I meant normalizing by $\sum_i w_i p_i$ (so that probabilities sum up to 1) rather than $\sum_i w_i$. Yes, in the corrected formula it is equivalent to $w_i n_i / (\sum_i w_i n_i)$. And I really recommend selecting $w_i$ such that $\tilde{p}_i$ it has some probabilistic interpretation (e.g. with this this money flow; or e.g. paying customers, or - customers who do something). Otherwise the choice of $w_i$ may be too subjective & hard to interpret. Commented Jan 21, 2016 at 9:54
• @H.Z. For Stack Exchange it may be not ideal to update question with an answer (now may answer makes little sense, as it repeats what you already said). For "does it makes sense": 1. What is your goal? Showing it to management, using it in machine learning, predicting it, etc? 2. In general, Renyi entropy makes sense only when $\tilde{p}_i$ can be interpreted (in some way) as probabilities. Commented Jan 21, 2016 at 9:58
• Well, I want this function to be more sensitive to countries with higher weights. So, if a country, say the i-th one, has a higher $w_in_i$, I want it to contribute to the output value more. I think that should be adjusted by choosing a proper value for $q$. Am I right? Commented Jan 21, 2016 at 17:49