I ran a simple online survey to ask 249 people which of three donation requests they found most compelling. This survey was slightly improved by randomizing the order of the answers.
Message A received 116 votes, message B received 49, and message C received 85. My first guess would be to identify the 95% confidence intervals for the various population proportions (e.g., 46.2% +/- 0.5% is still higher than 34.1% +/- 0.3%, etc) and choose a winner if its lower bound is higher than any competitor's upper bound, but something doesn't feel quite right about this.
One other thing: the messages asked the users to donate a dollar to a charity. We advised in the survey that 10% of the respondents would be offered a $1 bonus. The respondents were then asked to tell us whether or not they'd be willing to donate that dollar to the charity if they were selected. Of the 249 respondents, 174 gave us permission to donate their funds on their behalf. Filtering the results for only users who gave us permission to donate their funds does not appear to materially change how people answered but I wonder if it would be an interesting cross-reference point?
Any thoughts or feedback?