Percentile calculation for binary voting system Consider a system like Tinder for example where users can vote yes or no to a profile (like or dislike).  If Tinder wanted to come up with an internal percentile rating of a user based only on the other users' votes of their profile (not based on how picky that particular user is), what would be a good statistical approach?  
I have thought a good amount about this and I think there are two important points to consider.
1) Different users have different styles of voting.  Some users are pickier than others.  Thus, a like from a picky user should impact a profile's percentile more positively than a like from a less picky user.  The same logic goes for dislikes.
2) Votes more likely to be spam should be weighted less heavily in the percentile.  Spam can be determined by somehow analyzing a user's voting style and compare each vote they do to what everyone else is voting for a specific profile.
Beyond this I'm struggling to come up with a good algorithm to do this.
One other issue I'm thinking about is that let's say you come up with some number that represents a user's score in the system.  You can get this user's percentile by getting the ranking of that user's score among all the scores.  However, what if your system only has a small number of users?  The system I am thinking about is supposed to predict globally what the profile's percentile would be.  Thus, if there are only 5 profiles on the system and it is the top profile, this will only be the top 20th percentile. This would only be a problem when the system only has a few users, but I am wondering how to represent some idea of a percentile in this case.
 A: I do not have the one true answer to your problem, but FWIW I think you should look into Item Response Theory (IRT). That was, off course, developed for questionnaires but there are a lot of similiarities to your situation: In a Rasch model, each item has a difficulty and each participant decides whether for his particular situation, he decides to tick yes or no. In your Tinder example, a profile has an attractiveness and the user decides, whether with his particular standards is going to click yes or no. So looking into Rasch models or more broadly IRT you will find som probabilistic methods to determine latent traits.
I am not aware, whether you will find a way to weigh answers that are likely to be spam, though, but you will find methoeds, algorithms and statistics packages to compute how picky each user is and how attractive each profile is at the same time.
As always: You cannot do some statistical methods based on very small numbers. There will always be minimal requirements for the amounts of data to draw reliable conclusions.
