Suppose we have $N$ users that can vote for a given question by "Yes" or "No". Each user have a probability which is considered as the confidence that we have in that user (e.g. a user may have a probability of 0.8 to answer the question correctly, and a probability of 0.2 to answer it wrongly). Each user requesst an amount of money as a reward for answering the question. I (as the asker of the question) have a limited budget for paying the users.
Given this budget, which users should I ask to maximize the final answer to be correct (i.e. the answer obtained after aggregating the asked answers)?
"what does it mean for a vote to be correct": if the question is "does 10/2 equals 5?" then "Yes" is a correct answer to this question; now if the probability associated to the user is 1, that means that the user will answer this question correctly because it have a very good knowledge of the domain of the question (mathematics); if the probability associated to the user is 0, this means that the user will answer this question wrongly; if the probability associated to the user is 0.5, then his answer may be correct ("Yes 10/2=5") with a probability of 0.5, or maybe wrong wt probability 1-0.5.
What I want is to maximize the fact of getting the good final answer by aggregating the obtained answers (want to know that the answer to my question is "Yes" 10/2=5). For this, which users should I ask (given my budget, and given that each user request a given amount of money to accept to answer my question), and how to aggregate their answers in order to have the correct one.
What is called "vote" is the classical aggregation of the answers, by taking the most common answer among the received ones (if there is more "Yes" answers then ok I will consider the final result as "Yes), but in my real example I have some constraints: probabilities associated to users, ammount of money requested by each user to answer my question, and the total amount of money that I have, so I need to also consider these constraints. If you want, forget about the word "vote" in the question, just tell me who to ask and how to aggregate the answers that I get.