The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
1answer
34 views

What is a good way to convert movie ratings data (rated from 1 to 5, 5 being the best) to pairwise preferences?

I am working with movielens 100k, which is a dataset with 100,000 ratings of movies, rated by 943 users on 1640 movies. I need to convert these ratings to pairwise preferences between movies for every ...
3
votes
2answers
55 views

Determining proper K value for Elo rating

For the question I previously posted here with a general question about elo ranking applied to a a bit weird sport. From the ranking POV it's a bit like marathon - you can have up to n players but - ...
2
votes
1answer
45 views

Ranking Data Comparison

I'm currently analyzing my experimental data and I have a question about comparing two variables. In the experiment I have a question which asks respondents to rank a set of pictures based on their ...
4
votes
1answer
53 views

ELO rating for non-pairing sport + serious math

I was considering sport disciplines for which there are multiple players at the event but rather than playing against each other, they do stuff, are assigned points and their final position is based ...
1
vote
0answers
9 views

Creating products overall score across different data sets

I've created product score mapping that gets ranking for product according to reviews from different websites and in different contextual categories. There are 3 contextual categories: eCommerce/ ...
1
vote
0answers
13 views

How significatly less accurate is information we get from a 3 point scale rating system compared to a 5 scale rating system?

We are trying to add a rating system in our website and we are not sure whether to implement a 5 star rating system or a 3 scale(like,ok, don't like). The pro for the 3 point scale is that users would ...
0
votes
0answers
40 views

methods to validate effectiveness of ranking system?

I seek to compare the effects of different rating-based ranking methods and examine which ranking would lead to higher sales (i.e., which is ranking is superior to increase customer conversion rate). ...
0
votes
1answer
32 views

How do I control for changes in rating methods in this study?

I'm performing a study where subjects rate their perceived, subjective intensity of a stimulus presented to them at different magnitudes. To submit a rating, they click on on a logarithmic scale ...
0
votes
0answers
34 views

Normalized strength of schedule ranking system in R

What is the process of creating a pairwise ranking system in R? Specifically looking for the iterative ranking of margin of victory determined by strength of schedule to normalize for opponents ...
0
votes
0answers
16 views

Identify top performing products

I have monthly periodic data (id, total_ratings, average_rating, scanned_date) of millions of products. I want to calculate products which have performed well(in terms of rating) over last quarter. ...
0
votes
2answers
31 views

Sorting products by reviews considering the number of reviews

I have a big amount of products and a lot of reviews for these products, including a rating of the product. My problem is that every product does not have the same amount of reviews. For example, one ...
1
vote
1answer
251 views

Accounting for “autocorrelation” in margin-based Elo ratings

I'm trying to incorporate margin-weighting into a simple Elo rating system for a sports league. Classic Elo only encompasses Win/Draw/Loss data but I want to further reward teams for winning by a ...
1
vote
1answer
135 views

acceptable ICC for 6-point Likert scale

I have pairs of pictures of participants making facial expressions (240 pairs per participant, 11 participants). Two raters agreed on a criterion and rated each one of the pairs on a scale from 1 to ...
8
votes
1answer
157 views

Why do Elo rating system use wrong update rule?

Elo rating system use a gradient descent minimization algorithm of the cross-entropy loss function between the expected and observed probability of an outcome in paired comparisons. We can write the ...
1
vote
1answer
102 views

How to rank rankings? Two factors - freq and rank but freq has to has much more weight

My respondents has to evaluate number of items (say, 20). Formerly they had to just check three that they liked (like/dislike) and I simply counted the most checked. I 'improved' survey by replacing ...
1
vote
1answer
53 views

measure for jobs/complaints ratio

I have a number of jobs in a database which can receive complaints. A job can receive several complaints and a job has an associated supplier. I would like to identify the worst performing suppliers. ...
1
vote
2answers
88 views

Likert Scale 3 2 1 0 1 2 3 possible?

In my questionnaire I want to ask people to score on a dimensional Likert-type scale their attitude toward a specific target group. E.g. People with .... are stupid.... intelligent. I want to use ...
0
votes
1answer
23 views

Compare two items when rated by ordinal variables

This question was spurred by one of those tech support sites that asks at the bottom "rate how helpful this article was" and allows you to rate the article 5 for very helpful and 1 for not helpful at ...
2
votes
3answers
173 views

Analysing data on importance ratings

I had following question in my questionnaire: Rate the following factors: price, quality, advertisement, brand, reference from 1 (very important) to 5 (least important) that may have influenced your ...
0
votes
0answers
54 views

Performance metric

Let's say I have two people, A and B, who are each performing a different task. They have an aptitude score which designates how well they perform any task; A has apitude score a and B has aptitude ...
1
vote
0answers
2k views

Transforming scores between different rating scales – Importance of response options

I'm working on a meta analysis of the mean of a single variable (using arithmetic mean as the effect size, rather than the relationship between two variables). The attitude I'm assessing has been ...
0
votes
0answers
81 views

How well do analyst ratings predict stock perfromance?

Consider a huge dataset of which the following is a characteristic sample: ...
2
votes
0answers
224 views

Confidence intervals for 6-point Likert scale

I was asked to compare certain ratings between two different groups. The distribution of ratings looks something like this: ...
0
votes
1answer
81 views

How to apply Laplace rule of succession for any possible value in a range?

I am looking to have People rate certain items on a scale of 0-10 based on those items perceived utility to the particular Person performing the rating. Fractional values are permitted. Giving no ...
1
vote
0answers
44 views

how to compare partial ratings of a set of data by different groups of participants?

So lets say I have N items and I want to rate their relevance (say on a 5-point likert scale: 1 being least and 5 being most relevant) to the "perception of beauty" and I have M number of ...
1
vote
1answer
29 views

Can I split my data into several groups to be rated by various participants?

I am conducting a study on a pool of data which consisted of 200 items. I am planning to use rating scales to see how these items relate to a concept (e.g. "how are the next items related to the ...
1
vote
0answers
445 views

Item ranking algorithm [closed]

I am developing a website with a lot of different items that can be reviewed and rated, although I need a solid ranking algorithm which I can sort after. The ratings will be performed by the user ...
3
votes
1answer
656 views

How to interpret the capped binomial deviance as rating model fit in two player games?

In two player games using rating algorithms like Glicko, the capped binomial deviance is being regarded as a measure of the model data fit. Does anyone know how to interpret this measure? What ...
1
vote
2answers
692 views

How can I estimate a “confidence” in a score / rating

I'm sure there's a more technical term for what I'm looking for, but I don't know what it is (it's not confidence intervals, I think) I'm trying to make a system where people get ratings (from 1 to 5 ...
5
votes
1answer
93 views

adjust rating based on number of experiences

Suppose I am using a restaurant rating as an explanatory variable in a regression. The rating is defined as $R=\frac{G}{G+B+N+S}$, where $G$ is good, $B$ is bad, $N$ is neutral and $S$ is silent. I ...
0
votes
1answer
246 views

Simplest test for consistency of a rating scale

I want to explain to a non-novice doctor the best way to validate consistency of a measure across a likert-scale. The variable describes an average dose across 5 levels of patients (lower to higher). ...
1
vote
0answers
38 views

Detecting and preventing collusion in ratings

I am not sure whether or not this problem can be solved through statistical methods, but here is the problem: I have 4 professional reference providers for a job candidate rating the candidate on a ...
5
votes
2answers
363 views

Fixed-sum question

I have a set of 10 items each with a weight between 0-1. A person can rate each of these 10 items with a score between 0-100, but is given, say, only 500 points to allocate among these 10 items The ...
2
votes
0answers
70 views

Relative scoring system without much effect from outliers

I am trying to assign scores to phone's specifications to calculate a final rating score for each phone. However, when I was scoring based on megapixels. I found that one of the phones 'A' has a 41MP ...
3
votes
1answer
62 views

Is there a data decay method that decays data based on how much new data it receives?

So here's the situation: There are n balls. Each ball has been voted on a scale of 1-N by an arbitrary number of people. Every day (or every week, or more), some/most/all of the previous voters and ...
1
vote
1answer
533 views

Is there a statistically good way to split a set of data into (generally) uneven groups?

So here's the scenario: I have about 100 or so "items", each assigned with a number and total guesses... this number is a guess of what people think the item is worth (in terms of money). This number ...
18
votes
4answers
2k views

How to get started with rating and ranking based on pairwise competition data?

I'm interesting in learning about how to rate and rank individuals in a group who only interact/compete in a pairwise fashion (i.e., systems like the ELO rating system for chess). Are there are any ...
0
votes
0answers
65 views

How to develop a smart sort ordering of list data using popularity/rating algorithms

Here is the problem. We have list of Job orders - each job has certain state associated with it, you can think of state as a situation that job is in, for example, a job that is said to be done or ...
1
vote
0answers
59 views

How to calculate ROC(Ag) in this sound perception study?

I am studying individual differences in sound perception where I ask participants to listen to a pair of sounds and judge whether they are the same or different. The answer choice and scoring are as ...
6
votes
1answer
180 views

Do variations of ELO system exist for non-mirror games?

The ELO rating system and later improvements like Glicko are specifically designed, as far as I know, for mirror games. That is, we assume that no player has a systematic advantage over another. In ...
6
votes
3answers
5k views

How to analyse a ranking and rating scale together?

In some market research consumers are asked to rank the features of a product based on priority. For example, Rank the following features for a device based on your priority (1 being the topmost ...
0
votes
1answer
262 views

How to determine the accuracy of a statement?

Scenario: Consider a statement, e.g. "This movie is an action" Then let people vote on that. 1-5 where 1 is "Not action", 3 is "Some action" and 5 is "Pure action". Question How does one determine ...
1
vote
2answers
176 views

Intersubject concordance for ratings vs rankings

In my system I ask $n$ people to rate [1,5] $m$ objects (e.g. films). I'd like to get a measure of the inter-subject concordance, i.e. how much the users agreed overall, not just on a particular ...
1
vote
3answers
3k views

What is the most effective way to combine different rating scales for the same data?

I have a collection of data which is inherently the same (would expect it to follow the same distribution) where some of it is rated with one scale of ordinal ratings and the rest of the data is rated ...
4
votes
2answers
777 views

Combining ratings from multiple raters of different accuracy

I have a data set of J raters who each give ratings to I objects (on say a scale of 1 to 5). My goal would be to construct some kind of overall rating for each i based on all of the raters' scores. ...
8
votes
1answer
302 views

How to fairly determine winners for a regional science fair?

I need help figuring out the correct way to calculate winners at our science fair. I don't want my ignorance of statistics & math to get in the way of a kid's chances of winning. (lots of ...
2
votes
3answers
229 views

Aggregate opinions to a single measure

I have a following situation in which 6 experts are asked to give opinion about a bottle of wine by rating it from 1 - 5 stars. In addition, somehow we have a measure of the informativeness of each ...
1
vote
1answer
72 views

How to determine if application acceptance is “fair”

I'm a part of a volunteer organization that organizes a bunch of events. For each event, members need to submit an application in order to attend the event. A lot of people want to go to these ...
1
vote
0answers
101 views

Influences of scored/weighted ratings

I am experimenting with a scheme for aggregating $N$ scored ratings. The gathered ratings themselves are integers, e.g. a chosen value between $1$ to $5$, and each rating $r_i$ is assigned with a ...
2
votes
2answers
622 views

How do I sort an ordinal list of user-generated ratings data?

The Ubuntu Software Center uses a 1 to 5 star rating system for its App Reviews. However, it's current ratings sorting algorithm looks very fishy. I believe this is a different question from this ...