How do I show correlation with the "fat head" and "long tail" of a distribution? I run an information booth at an airport. People come up to the booth all day and ask questions. Some questions get asked over and over again. (“Where is the rental car counter?”) Some questions are very rare. (“Can I bring a baby llama on an airplane?”) You can measure this as a frequency—maybe the number of times per week a given question gets asked. The distribution of question frequencies looks like a power law with a “fat head” of a few popular ones getting asked all the time and a “long tail” of many unpopular ones only every getting asked once or twice.
There are two agents working at the counter: Alice and Bob. People come up to them at random, so on average they answer the same number of questions. Each agent provides either a helpful answer on an unhelpful answer. So I collect a bunch of trials like so.
Question A    Frequency: 1     Agent: Alice   Outcome: Helpful
Question B    Frequency: 1     Agent: Alice   Outcome: Unhelpful
Question C    Frequency: 133   Agent: Bob     Outcome: Unhelpful
Question C    Frequency: 133   Agent: Alice   Outcome: Helpful
Question D    Frequency: 2     Agent: Bob     Outcome: Helpful
…and so on…

The helpful/unhelpful outcome is a deterministic function of the agent and the question (e.g. an agent always gives the same answer to any given question). I will naturally tend to have more trials involving frequent questions (e.g. I’ll have more trials involving question C than question A), though I have enough samples to get a fair representation of the low frequency ones.
I have a hypothesis that Alice is better at answering high-frequency “fat head” questions and Bob is better at answering low-frequency “long tail” ones. How could I use the data above to confirm or deny that hypothesis?
I think I want to show an interaction between the independent Agent and Frequency variables with response to the dependent Outcome one, but I’m not sure how to frame it beyond this, or which statistical test I should use.

So far I've come up with calling questions with frequency 1 "long tail" and all other questions "fat head", then doing a $\chi$-squared test for the interaction between this categorical variable and the agent with respect to the number of helpful answers. I don't like this, though, because I don't want to draw an arbitrary line through my frequency distribution.

Follow-on question: suppose for every question I am guaranteed to have an answer from both Alice and Bob. How does this change things?
 A: You could consider that only questions where the agents give different answers are informative (A helpful, B unhelpful, and vice versa). That reduces it to a problem where you have a binary outcome (A better than B versus B better than A) which you could then regress on frequency using logistic regression.
A: If you bin the questions by frequency, you can get something like the datapoints you're looking for. To use your example, for questions with frequency 1, Alice gets a 0.5, for questions with frequency 2, Bob gets a 1, for questions with frequency 133, Bob gets a 0 and Alice gets a 1.
You may want to first apply some sort of shrinkage--knowing that Alice was helpful for 5 out of 5 questions in a class is more useful than knowing that she was helpful for 1 out of 1 questions in a class. If you think helpfulness is something like a Bernoulli trial, then the Beta distrbution makes sense (which is simple--instead of $\frac{right}{total}$ you use $\frac{right+1}{total+2}$).
You may also want to do scale-sensitive binning--it might make sense to consider all frequencies above 100 as one group, for example, so that you can get a reasonable rate (it's  likely that multiple rare questions will have the same frequency, but unlikely that multiple common questions will have the same frequency).
You can now do a number of regression or classification techniques--a logistic regression, like suggested by mdewey, could be appropriate.
