How to calculate overall rankings/ratings from many pairwise comparisons? I have a number of categories and a bunch of data about pairwise comparisons between those two categories; how do I learn information about overall rankings/ratings/etc. of all the categories?
Let me provide more detail.  I'm trying to learn what the general population is most worried about, by surveying them.  I have about 20 categories of potential concerns, call them $A, B, ..., T$.  I have picked a random sample of people.  For each person in the sample I have done the following.  I picked a pair of concerns, e.g., $E$ and $Q$.  Then I asked the respondent: which are you more worried about, $E$ or $Q$?  They could select one of three possible answers: $E$, $Q$, or "about the same for both".  I have about 300 responses.  Each respondent got a different, independently random selection of a pair of concerns.
How should I analyze this data, to extract as much information as I can?  For instance, can I learn anything about what concerns people are generally most worried about, or least worried about?  Can I compute some approximate rankings or ratings or some comparisons?  And, of course, I am concerned about computing statistical significance for any conclusions drawn.  Any advice?
P.S. Feel free to choose any reasonable model for how people will respond.  For instance, you could assume transitivity for each individual (each person has an internal ranking of concerns, and answers the question based on that ranking; so if they would answer that they're more worried about $P$ than $Q$ if asked about $P$ vs $Q$, and would say they're more worried about $Q$ than $R$ if asked about $Q$ vs $R$, then if asked about $P$ vs $R$ they would say they're more worried about $P$ than $R$).
 A: Think of each potential concern as being a chess player. When E and Q are paired together and a judgment is made, then either player E wins, there is a draw, or  player Q wins. 
How can we determine the relative strength of two players who have never met, and are likely to never have had a common opponent?
This problem is solved in chess by Elo type models; see more recent work by Glickman. http://www.glicko.net/research.html  There is no requirement that anywhere near all the possible pairs have been directly evaluated. 
You can also just start with the Wikipedia page, which isn't bad. 
http://en.wikipedia.org/wiki/Elo_rating_system
The Elo model also appears in the movie Social Network -- it's the equation scribbled on the window that is used to determine the relative hotness of various women.
A: This highlights the importance of choosing an analysis plan before designing the data collection instrument. The analysis plan needs to drive the survey format, not vice versa. While you have intuitively landed on a discrete choice methodology, which is often very useful in answering these questions, I think the design isn't quite sufficient to answer the question you have.
If you had asked how to design a survey to answer this type of question, I would have told you that "Max Diff" (short for maximum difference) is one of the popular ways to do this. In Max Diff, sets of ~5 concerns would be presented together and the respondent would be asked to choose the most and least important. Then a standard scoring methodology is used to provide a group-level score of the relative importance of the various attributes. 
One of the reasons that Max Diff is popular is that making responses on a set of attributes tells you about multiple pairwise comparisons, and thus allows you to avoid asking 190 questions and will help you get a higher survey completion rate.
While many statisticians have spent many hours developing strategies to reduce the number of questions that need to be asked, I know that respondents are still required to respond to a series of sets in order to understand how the importance of each item stands relative to the others. For that reason I think the data you have is not going to be sufficient - a single pairwise comparison for each respondent.
