# Questions tagged [rating]

"A rating is the evaluation or assessment of something, in terms of quality (as with a critic rating a novel), quantity (as with an athlete being rated by his or her statistics), or some combination of both" (Wikipedia).

161 questions
Filter by
Sorted by
Tagged with
76 views

### Computing ICCs in Matlab, to assess rater consistency (inter-rater agreement)

In an experiment, participants rated each of several musical stimuli on 5 aspects (ratings A to E), across ~3,000 trials. Stimuli were performed (by a previous set of subjects) either solo or duo (...
• 1,570
62 views

### Interaction between variables to create a ranking system

I was wondering if there is a standard way to rank cases by multiple variables. When we have only 1 quantitative variable, the ranking is trivial, you sort the cases by comparing the magnitude between ...
292 views

### help with formula to calculate Bayesian ranking of M-star reviews

I wonder, if someone could help to understand a formula from a book please. Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference (Addison-Wesley Data & Analytics) (...
• 1,316
172 views

### Can you do hypothesis testing when instead of a 'sample' size you have 'actual' size? Alternatively, how would you use statistics to compare means?

From chess se: What is white's increased advantage in chess90 as compared to chess870? (Chess960 can be split into 2 subsets, chess90 and chess870) Intro: It is known in standard chess that white has ...
• 2,176
1 vote
312 views

### Why are rating distributions 'smoother' when there are more players?

Consider the following rating distributions by variant/time control on lichess. The bullet, blitz and rapid rating distributions look very, well, 'smooth' (or holomorphic/analytic or harmonic or ...
• 2,176
24 views

### Is it appropriate to use difference scores in this context?

I am trying to compare the level of rating accuracy of two groups of participants (say Sample A and B). The study design is as is: I asked a consumer panel to rate how much they liked three products. ...
• 1
140 views

### Prove or disprove that rating worth in online chess depends on the time of the day

Crosspost from Chess.SE. TL;DR I feel that ratings on a chess website are worth differently depending on the time of the day, and I want to validate the claim. I have a complete history of games ...
• 131
1 vote
21 views

### Comparing ratings & statistical difference

I have a dataset where I'm comparing overall ratings between multiple "products." It looks something like this: Product 1: 3.5 Product 2: 4.1 Product 3: 3.7 ... Product 7: 3.3 What I'm doing ...
• 111
30 views

### Tests for item rating reliability (Cronbach's alpha?)

Here is the situation: I am asking participants to provide some names of a certain category. (Imagine a verbal fluency task where you are asked to name as many animals as you can in the space of 1 ...
60 views

### Q: Aggregating audience and critic movie ratings, and creating a composite rating of the two

I’m building a movie review aggregation site that combines user ratings and critical reviews for a given movie. The objective is to create a list of the “best of the best” movies which were rated ...
1 vote
43 views

### Weighted Kappa with rating variables that have different set of categories

I want to measure the inter-agreement rate between two physicians that rated the same set of images with two different methods (rater1 used method1, rater2 used method2) BUT Rater1 rated the the ...
• 457
1 vote
931 views

### Ordinal variable in multiple linear regression model?

I am estimating the price of a car using past data (in other words, Predictive Multiple Linear Regression) and one of the variables is a rating from 1 to 4 of a particular guideline. How do I treat ...
26 views

• 432
1 vote
399 views

### How to better rank based on ratings that takes the number of reviews into account?

Normally on most websites when items are being ranked based on reviews, they’re simply based on the average rating (out of 5 or 10); however, I believe an item that has an average rating of 4 based on ...
38 views

### What is the name of this test for the probability of differences between lists and is it valid?

And are there any better methods? In a 1969 book about investing, the author Burton P. Fabricand describes a method for assessing the effectiveness of a stock picker. Ask the picker to provide a list ...
• 449
86 views

### Turning a player's points value into a 1-10 match rating scale? [closed]

So I have the players "fantasy score" stat. This stat is the sum of their ingame stats multiplied by preset weights. These weights are from the video game DOTA 2's official fantasy points weights. For ...
249 views

### Elo-type ranking system that incorporates game score

The Elo rating system is used to calculate relative skill levels between individuals or teams. It can be applied to many types of games and sports, but when applied, it only considers wins and losses. ...
• 1,042
359 views

### comparing bipolar scale to unipolar scale

I'm trying to build some surveys and I want to ask the following questions: After 3 weeks, how do you feel about decision to join our company? Very Dissatisfied Moderately Dissatisfied Slightly ...
• 209
115 views

### best-worst ranking: do all items have to be blocked together at least 1 time?

Here there are instructions to build Max-diff (or Best-worst) ranking questionnaire: In an ideal world, a MaxDiff experimental design has the following characteristics, where each alternative appears:...
470 views

### How to decide the number of options to give to people in a poll? (for opinion based, subjective contexts)

Just to be clear, this question is open ended on purpose, I am fine with(or even prefer to) being pointed to a few research papers, and proper areas related to this subject. I am not sure what I am ...
• 33
48 views

### Averaging bang for the buck

I'm building a table/graph to compare multiple video cards according to a given benchmark. The benchmark consists of multiple measurements using games. For each benchmark measurement, I build a ...
• 101
1 vote
100 views

### Which rating variable is affecting the main rating?

I have 3 types of rating, the restaurant rating (main rating), food rating, and service rating. Each of these ratings go from 0 to 2. I would like to know if it's possible to tell if a restaurant is ...
• 111
115 views

### Considering sample size when comparing average ratings

Consider, for example, an online shop listing multiple products available for purchase. Each product has an average rating from 1 to 5 calculated from all that product's customer ratings to date. To ...
• 21
1k views

### General principles for extending the Elo system to games in which the margin of victory matters

Note I am a first-time user of this site, and am not sure what determines how broad a question is. I consciously limited the question to general principles in order to avoid making it too broad (by ...
464 views

### Does Elo rating system address isolated groups of players?

I am looking to apply a flavor of the Elo Rating System to a game that is played all over the world (not chess) and am toying with the idea of adding what I might consider an "enhancement" to it. So ...
• 143
1 vote
74 views

### One sample test for helpfulness ratings

I have some questionnaire data were participants rated with [0, 1, 2, 3, 4] whether a product helped with respect to several attributes. 0 indicates that it didn't help at all and 4 is very much. I ...
33 views

### Magnifying the outliers in a rating system to rank objects better

1) I am trying to design a product for weekly ratings (1-5) of an entity. The entity's rating will be reset every year. I know most of the weeks the rating will be the same (3) and I want a way to ...
42 views

### Are average online ratings of products (say out of 5) assumed to be on a linear scale as understood by the user?

Suppose 2 washing machines A & B are rated as 3.5 and 4.0 (out of 5) respectively, vs another 2, C & D, that are rated as 4.2 and 4.7 respectively. Can I make some useful inference based on ...
• 175
137 views

### How much information is gained when sampling from a random distribution?

This is a problem I have encountered while considering ELO ratings for a game like Go. Suppose we have two players, one has a known rating, the other's is unknown but assumed to be close to the first....
59 views

### How can I rate the predictive power of a function through a heatmap?

Context: I use a certain function that is supposed to estimate the possibility of an event "B" to occur in a given area, following the occurence of event "A". The input is a map of the area, and the ...
• 43