I have many users, each of whom may or may not have had the opportunity to perform each of various actions and may or may not have performed it. That is, for each action, some users had the opportunity to perform it and some did not; among those who had that opportunity, some did so and some opted not to.

Moreover, those actions are in groups, and I want to associate users with the various groups.

I want to see which users opted for which action more often than they would be expected to (and, ideally, how much more often). But "they would be expected to" includes only users who had the opportunity to perform the action.

Here's an example: There are a million people. There are thousand shops. Each person may or may not have been outside the entrance of each shop, which I'll call "near" for brevity. If he did so, he may or may not have entered the shop. I have the data on which user was near which shop, and which user entered which shop. (I don't care how many times he entered, just whether he ever did.)

There are also one hundred categories for shops, and each shop may be in more than one category (e.g., "bakery", "part of a chain of stores", "dimly lit"). (Each shop is in at least one category, say.)

I want to know, for each user, which categories of shop he chooses to enter unexpectedly often. I must take exclude from my calculations all shops he has never been near (because his failure to enter those shops was, let's say for this example's sake, not by choice).

For something like this, normally, I'd think to use a contingency table with a $\chi^2$ independence test (and, post hoc, see which residuals are greater than some cutoff). But here that would mean that one dimension of the table will be the various users, the other will be the various actions, and the values will be $1$ or $0$ depending as whether the user performed the action. I'm very loath to do that here, for three reasons: (1) there will be a lot of missing data, representing users who did not have the opportunity to perform the action, and I wouldn't know how to count them in the $\chi^2$ except as $0$, which of course would mean they opted not to perform the action; (2) I think the expected values will typically be much less than $1$ and most of the observed values will be $0$, and despite the large number of cells I don't think the $\chi^2$ will be high enough to overcome the numerous degrees of freedom and thus the test is bound to fail (though maybe I can use Fisher's exact test instead?); (3) it seems odd to me to use a $\chi^2$ when the table contents are binary rather than counts.

It may possible to get around some of the above objections by using rates instead of counts, so that (in the example) the expected value for each cell in the contingency table should be (how many shops the user entered ÷ how many he was near)×(how many users entered this shop ÷ how many were near it)÷(the total number of user-shop combinations for entering ÷ the total number of user-shop combinations for being near), but is that a reasonable number to look at? (Also, then the usual $\frac{\left(O-E\right)^2}{E}$ wouldn't accurately reflect unexpectedness, because the numbers are bounded between $0$ and $1$.) And, anyway, there are more issues:

If I do use a $\chi^2$ test as described above, that only gives me information on individual actions (shops in my example) and not categories of actions, which is what I want to know. Nor can I simply combine the data by category and see how often each user entered each category of shop, because that ignores the differences in popularity among individual shop. (If the "dimly lit" category has only two shops, say, one of which is more popular than the other, then their expected entrance counts will be very different; if use $A$ has only passed the one shop and user $B$ has only passed the other and we combine entrance counts for the category, we lose that information.)

So what should I do instead? (I'm certainly open to methods not including a contingency table.)

  • $\begingroup$ Read David Dunson's papers on massive contingency table models using tensors. $\endgroup$
    – user78229
    Mar 30, 2017 at 15:16


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.