0
$\begingroup$

Suppose the following dataset:

        | Region 1 | Region 2
------------------------------
Party 1 |   20%    |   10%
------------------------------
Party 2 |   20%    |   30%
------------------------------
Party 3 |   60%    |   60%
------------------------------

Let's suppose there's been an election in two regions (i.e. different people voting) for three political parties 1, 2 and 3 (same parties in both regions) and the fraction of the votes is as above.

Question:

Is it reasonable and correct to compute a Pearson's Correlation Coefficient between two regions to asses whether the people in the regions voted similarly? Would Spearman's rho or Kendall's tau make any difference?

I have pretty strong intuition and, so to speak, soft arguments that it might be inappropriate and something like Cramér's V should be used instead, but I'm looking for some theory/statistics based explanation.

$\endgroup$
1
  • $\begingroup$ Both LiKao and Lewian provided awesome answers, but I can only accept one. I suggest anyone interested in the topic more to read the comments as well. $\endgroup$ – Matek Feb 11 '20 at 20:56
2
$\begingroup$

There is no unique answer to this. It depends on what you mean by "similarity", which one could say is implicitly defined by the measure you choose. Note that the data alone or statistical theory cannot tell you what "similarity" you should be interested in!

Personally, for the given situation, I would not use Spearman or Kendall correlation, because this loses the precise quantitative information in your values, which I think is important (particularly if you don't have that many parties).

Neither would I use Cramer's $V$, because this is equivalent to the $\chi^2$-test, which relies heavily on the underlying number of observations $n$, and I don't think that you'd rate the difference between your percentages differently if that was based on 100,000 rather than 100 people (the other answer made me think though that if you want a significance test of equality, this would actually be fine; I just don't think that in your example this makes much sense, because there is no reason why perfect equality should hold, and with enough voters you will always find significance).

Pearson correlation doesn't look obviously bad to me at first sight, although I wonder whether you could just compute the $L_1$-distance between the proportion vectors, i.e., summing up the componentwise differences. That gives you a distance measure (or 100% minus that a similarity measure) that is very straight and simple to interpret. The thing with Pearson correlation is that this is in the first place useful for detecting linear dependence, but here the only possibility of having linear dependence is equality, so the situation is quite restricted compared to the one Pearson is made for.

Another approach to deal with such "compositional" data is so-called Aitchison geometry. For computing a dissimilarity this would amount to computing a distance between log-ratio transforms, see here. John Aitchison has motivated this through some axioms, although I believe that these are not very relevant to the situation that you outline here. Personally I believe that this approach would give you something that is too dominated by the differences between small proportions, a well known problem with Aitchison's approach.

$\endgroup$
8
  • $\begingroup$ Not sure if L1 distance would be such a great distance score here. To get mathematical, I would use the fact that percentages "live" on a N-simplex and the use the L2 distance along the simplex (or rather 1-d). So essentially just leaving out one of the numbers and then calculate the distance. But in general, it seems we had the same idea in our answers. $\endgroup$ – LiKao Feb 11 '20 at 15:03
  • $\begingroup$ One property that straight $L_1$ has is that if 10% voters change from one party to another, the change in distance is constant regardless of which parties these are and how big they were before. This I find a reasonable property, and I'm not sure whether this can be achieved reducing information to the simplex. $\endgroup$ – Lewian Feb 11 '20 at 15:15
  • $\begingroup$ Thanks, thast's helpful. I have some follow-up question though. The observations within a variable are not independent, as they all sum up to 100% (so e.g. knowing n-1 observation values, you know the last one's value so it's not "free"). Isn't this a problem or some assumption violation for Pearson's coefficient? $\endgroup$ – Matek Feb 11 '20 at 15:18
  • $\begingroup$ It's not an assumption violation for computing and interpreting Pearson correlation; however it is an assumption violation if you want to compute tests and confidence intervals based on it. (In that case a $\chi^2$-test would probably be appropriate.) $\endgroup$ – Lewian Feb 11 '20 at 15:20
  • 1
    $\begingroup$ As I said. The assumptions are assumptions for the distribution theory behind tests and confidence intervals, not for the definition of the Pearson correlation. Having seen lots of data sets and their correlation values, I feel pretty confident about interpreting the raw values, but you may not. Obviously this also depends on what your analysis is used for and how and to whom it is communicated. $\endgroup$ – Lewian Feb 11 '20 at 15:31
2
$\begingroup$

The most general answer: It depends.

The crucial question is, what exactly do you mean by "voted similar". In statistics, often people tend to just use some given measure and then assume that it fits what they are testing. But it is important to first clarify what the terms mean.

As a somewhat sophisticated example. Assume Party 1 and Party 2 are two types of left-wing parties, and Party 3 is a conservative party. If you want to test if the political climate in the two states is similar, then surely differences between voter outcomes between Party 1 and Party 2 don't matter as much as differences in the outcomes between 3 and 1 or 2.

If you are just trying to find something descriptive, however, and don't care about deeper meaning of your concept, then the Pearson correlation can be a great tool, even when dealing with percentages (but beware the points below).

Most often, however, when people ask if such-and-such is correct then they are asking about inferences from these measures. So you might want to know if the p-value for a Pearson correlation on this data makes any sense.

Here are some key points, that might help you to decide this yourself (these apply to p-values from Pearson correlations):

  • How many degrees of freedom does your data have (the data, not the correlation coefficient)? You might be tempted to say three, because there are three numbers. But are there really three independent measurements?
  • Also, Pearson Correlations assumes normality. Can percentages ever be normal distributed?

Addendum:

Since there seems to be some confusion on why the Pearson Correlation might still be useful in this case, here a simple example.

Assume you clarify your question in the following way: "Given two random voters ($V_1$ and $V_2$), one from each state, what is the probability that both voted for the same party". This would give a simple measure of similarity, that also has a straight forward interpretation.

Let's do the calculation:

Assume the ratio of people who voted for each party in state 1 is given as $p_{1,1},\ldots,p_{N,1}$ (there are $N$ parties) and in state 2 it is given as $p_{1,2},\ldots,p_{N,2}$. The probability that $V_1$ voted for the first party is $p_{1,1}$ and likewise for all other parties. If $V_1$ voted first party, the probability that $V_2$ also did it (conditional probability) is $p_{1,2}$. Because those two voters can be assumed to be independent, the joint probability is $p_{1,1}\cdot p_{1,2}$. We have to sum up over all possible votes for $V_1$, so we get

$\rho = \sum_{i=1}^N p_{i,1}\cdot p_{i,2}$.

This is looking quite similar to a correlation coefficient, isn't it?

$\endgroup$
7
  • $\begingroup$ Computing and interpreting Pearson correlation does not assume normality; only if you start to compute p-values and confidence intervals you need that assumption, and approximately it still can be fine for percentages. One issue with these data, particularly if there are not many parties, is that the percentages are negatively dependent among each other (e.g., if you have a big one, the others need to be smaller), which standard correlation tests would not allow for. $\endgroup$ – Lewian Feb 11 '20 at 15:01
  • $\begingroup$ @Lewian Yup, agreed. The r in itself has no reason to require normality and hence can be informative even if it is violated. I thought because I explicitely mentioned p-values in the preceding paragraph, it was clear that these two points only relate to interpreting p-values from PMCC. $\endgroup$ – LiKao Feb 11 '20 at 15:05
  • $\begingroup$ @LiKao I don't quite feel the difference between something being descriptive/interpretable and making inferences from it, unfortunately. Does this relate strictly to using vs not using the statistical tools (p-values, confidence intervals, tests) that require the mentioned assumptions to be fulfilled? $\endgroup$ – Matek Feb 11 '20 at 15:41
  • $\begingroup$ Is it like predicting values using linear regression not caring about anything else (assumptions not required) vs interpreting lr's coefficients, p-values, etc (assumptions required)? $\endgroup$ – Matek Feb 11 '20 at 15:49
  • 1
    $\begingroup$ What null hypothesis would you want to test? The parameters of what model would you want to have a confidence interval for? (If the answer is "nothing" and "none", I don't see why you would worry about the assumptions for such methods.) The difference is between measuring similarity or distance, and doing statistical inference about certain parameters. Your question doesn't suggest that you want the latter, although you still may. $\endgroup$ – Lewian Feb 11 '20 at 15:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.