Relationship between categorical factors

Question

I am not sure what this is called in English, but if we have two categorical factors, we can say that one of them (A) is finer than the other (B) if it holds true that if two observations belong to the same A group, then they'll also belong to the same B group.

An example is countries and cities. If two observations are from the same city, certainly they are also in the same country. So City is a finer grouping than Country.

Question: Given a dataset where for each observation we've indicated what group it belongs in for a number of factors $A,B,C,...$, is there a quick way to check whether one of the factors is finer than the other? An algorithm, some code, or something? Right now, I literally have to sit down and check it by hand, and when there are many factors, it's a slow process.

Answer 1

You have defined "finer" as "if $A$ then $B$." In the propositional calculus, using 0 for false and 1 for true, "not finer" therefore has the truth table

A  B  
-- -- --
0  0  0
0  1  0
1  0  1
1  1  0

Arithmetically this is equivalent to A * !B where negation, !, reverses zeros and ones. Accordingly, given an indicator vector, we can count the number of places where $A$ is not finer than $B$ by summing these values. That sum of products is precisely a matrix multiplication. Thus, if you were to bind all your indicator vectors for $A, B, C, \ldots$ into a matrix $X$, then the zero non-diagonal entries of the product

$$X^\prime (!X)$$

would indicate the "not finer" relation and (therefore) the zero non-diagonal entries would show you which variables are finer than which others. This is a combination of two extremely fast operations, negation and matrix multiplication, and so is practically effective even on huge datasets and large numbers of variables.

---

Example

Here is code in R to generate five variables in which $A$ is finer than $C$, $D$ is finer than $B$, and $E$ is finer than $C$.

set.seed(17)
n <- 80
a <- runif(n) < 1/2
b <- runif(n) < 1/2
c <- pmax(a, runif(n) < 1/2)
d <- pmax(b, runif(n) < 1/2)
e <- pmax(c, runif(n) < 1/2)

The code for $X^\prime(!X)$ is

x <- cbind(a,b,c,d,e)
y <- crossprod(x, !x)
z <- y==0
diag(z) <- NA
matrix(colnames(x)[which(z,arr.ind=TRUE)],ncol=2,dimnames=list(NULL, c("child","parent")))

Its output is

     child parent
[1,] "a"   "c"   
[2,] "b"   "d"   
[3,] "a"   "e"   
[4,] "c"   "e"

It has correctly picked up the fact that $A$ is finer than $E$ because $A$ is finer than $C$ is finer than $E$.

Because errors might crop up in data, it might be better to inspect the actual counts and screen not for zeros but for very low counts. To that end, this code has preserved the matrix of counts in y, which looks like this:

   a  b  c  d e
a  0 19  0  8 0
b 20  0 11  0 6
c 19 29  0 12 0
d 32 23 17  0 8
e 29 34 10 13 0

It is now clear that the other variables aren't even close to satisfying a "finer" relationship: the nonzero, non-diagonal counts are all relatively large. For instance, there are 6 exceptions to the assertion "$B$ is finer than $E$."

Set n <- 1e7 to generate ten million observations and run this again. You will get the correct results, computed in about one second.

Answer 2

Suppose your dataset has two factors A and B and you want to check whether or not A is finer than B.

So this R script should do the trick, using dplyr package

dataset %>% 
    group_by(A) %>% 
    summarise(unique.B = (n_distinct(B)==1)) %>% 
    select(unique.B) %>%
    all()