The Problem:

I have measured two binary variables within 1 categorical variable with 5 levels.

Initially, I thought I'd be able to use Fisher's Exact test or some $N \times M \times K$ version of it. However I have only found references where the dimensions of the table are specifically $2 \times 2 \times 2$ (see the "hypergea" package in R as an example). In my case it is a $5 \times 2 \times 2$.

I was wondering if there are other approaches that would be appropriate.

Here's an example of what I'm working with in R.



I'll note here that Var1_ and Var2_ were measured from different individuals, though the individuals were sampled from the same population. That's why I placed the NA's in the data frame. Also, note that some of the data may be pseudoreplicated by ID - I'll come back to that later.

I'm looking for a way of analyzing this problem.

Specifically I want to know:

1. Does the state of Var1_ significantly predict the state of Var2_? What is the relationship and how strong is it?

2. Is this relationship significantly different between GROUP levels? How different is it (effect size)?

My approach so far:

Some searching indicated that a GLM might be appropriate. However, as far as I can tell, the GLM function requires the V1 and V2 variables to be observations from the same sample. So, just to see how it looks, I'll take the NA's out and run a GLM.



glm_obj_1 <- glm(Var1_ ~ Var2_, data=testdata, family=binomial(link="logit"))


Ok... I think that might be what I'm after, but I'd like to know if the relationship is different between GROUPs.

glm_obj_2 <- glm(Var1_ ~ Var2_* GROUP, data=testdata, family=binomial(link="logit"))

Ok! Is this a reasonable approach? Removing the NA's seems questionable to me, but I'm not sure how else to deal with that in the GLM specification.

Extra Credit:

I mentioned the pseudoreplication before. I assume this could be controlled for with a random effect? I'll try a GLMM implemented in LME4... This time I'll add fewer ID levels to increase the odds of pseudoreplication.



glmer_obj<-glmer(Var1_ ~ Var2_ * GROUP + (1|ID), data=testdata, family=binomial(link="logit"))


Well, it fails to converge, but for illustrative purposes, I hope it suffices. Would this be an appropriate way to tackle this type of data?

I'm eager to hear your responses, suggestions, criticisms, etc.

  • $\begingroup$ How about Cochran–Mantel–Haenszel statistics ? $\endgroup$
    – MJW
    Apr 19, 2017 at 10:22
  • 1
    $\begingroup$ Every ID is one unit (e..g, person), right? It seems like you don't have any units for which both Var1_ and Var2_ were observed: Then I don't think that you can say much about their relationship... $\endgroup$ Apr 19, 2017 at 12:39
  • 2
    $\begingroup$ @hplieninger That's what I was afraid of. Though, I don't understand why. A concrete example: You measure the prevalence of smoking (1= smoke, 0 = don't smoke) among 5 age groups. Each group has a different probability of smoking. Then you measure the prevalence of lung cancer (again, assume a binary response) in the same age groups, though in different individuals. Aren't they both estimating the value of a trait of each of those populations? Why must the data come from the same people if we know the people were sampled from the same population? $\endgroup$
    – baffled
    Apr 23, 2017 at 10:52
  • 4
    $\begingroup$ Because the relationship at the group level may or may not be the same as the relationship at the individual level (which you are probably interested in). See ecological fallacy. $\endgroup$ Apr 24, 2017 at 10:41
  • 2
    $\begingroup$ I think it would help if your question was more concrete. What are these groups? experimental groups? natural groups? Why is either Var1_ OR Var2_ observed? Why are some IDs measured more than once and others not. $\endgroup$ Apr 24, 2017 at 10:46

1 Answer 1


I will try to simplify your question, and consider these simplifications.

If we ignore the group (and only consider group 1). You question is if there is any relation between var1 and var2. As "hplieninger" comments above there is no way to compare the variables directly if you dont have measurements for both var1 and var2 for the same individual.

However, a practical solution, though not ideal, would be to use matching. In other words you match an individual with var1 with an individual with var2. Thus you get a dataset of 25 matches in each group. Hereafter you pretend that the matches is indeed the same person, and make a regression model. I will not go into details about matching here, but be careful about conclusion following this metod.

Another way of simplifying is to consider only var1 and group. In this way you can use simple tests or a regression method to evaluate the association between group and var1. Similar with var2. However this does not consider any relation between var 1 and var2. You can evaluate results to interpretate if var1 and var2 seems to have similar tendency across group, however note that this is only interpretation with not statistical evidence.

I hope this might help you forward in you process.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.