Questions tagged [rating]

"A rating is the evaluation or assessment of something, in terms of quality (as with a critic rating a novel), quantity (as with an athlete being rated by his or her statistics), or some combination of both" (Wikipedia).

Filter by
Sorted by
Tagged with
0 votes
0 answers
76 views

Computing ICCs in Matlab, to assess rater consistency (inter-rater agreement)

In an experiment, participants rated each of several musical stimuli on 5 aspects (ratings A to E), across ~3,000 trials. Stimuli were performed (by a previous set of subjects) either solo or duo (...
user avatar
  • 1,570
0 votes
0 answers
62 views

Interaction between variables to create a ranking system

I was wondering if there is a standard way to rank cases by multiple variables. When we have only 1 quantitative variable, the ranking is trivial, you sort the cases by comparing the magnitude between ...
user avatar
4 votes
1 answer
292 views

help with formula to calculate Bayesian ranking of M-star reviews

I wonder, if someone could help to understand a formula from a book please. Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference (Addison-Wesley Data & Analytics) (...
user avatar
  • 1,316
0 votes
1 answer
172 views

Can you do hypothesis testing when instead of a 'sample' size you have 'actual' size? Alternatively, how would you use statistics to compare means?

From chess se: What is white's increased advantage in chess90 as compared to chess870? (Chess960 can be split into 2 subsets, chess90 and chess870) Intro: It is known in standard chess that white has ...
user avatar
  • 2,176
1 vote
2 answers
312 views

Why are rating distributions 'smoother' when there are more players?

Consider the following rating distributions by variant/time control on lichess. The bullet, blitz and rapid rating distributions look very, well, 'smooth' (or holomorphic/analytic or harmonic or ...
user avatar
  • 2,176
0 votes
0 answers
24 views

Is it appropriate to use difference scores in this context?

I am trying to compare the level of rating accuracy of two groups of participants (say Sample A and B). The study design is as is: I asked a consumer panel to rate how much they liked three products. ...
user avatar
  • 1
3 votes
1 answer
140 views

Prove or disprove that rating worth in online chess depends on the time of the day

Crosspost from Chess.SE. TL;DR I feel that ratings on a chess website are worth differently depending on the time of the day, and I want to validate the claim. I have a complete history of games ...
user avatar
1 vote
1 answer
21 views

Comparing ratings & statistical difference

I have a dataset where I'm comparing overall ratings between multiple "products." It looks something like this: Product 1: 3.5 Product 2: 4.1 Product 3: 3.7 ... Product 7: 3.3 What I'm doing ...
user avatar
  • 111
0 votes
0 answers
30 views

Tests for item rating reliability (Cronbach's alpha?)

Here is the situation: I am asking participants to provide some names of a certain category. (Imagine a verbal fluency task where you are asked to name as many animals as you can in the space of 1 ...
user avatar
0 votes
1 answer
60 views

Q: Aggregating audience and critic movie ratings, and creating a composite rating of the two

I’m building a movie review aggregation site that combines user ratings and critical reviews for a given movie. The objective is to create a list of the “best of the best” movies which were rated ...
user avatar
1 vote
1 answer
43 views

Weighted Kappa with rating variables that have different set of categories

I want to measure the inter-agreement rate between two physicians that rated the same set of images with two different methods (rater1 used method1, rater2 used method2) BUT Rater1 rated the the ...
user avatar
  • 457
1 vote
1 answer
931 views

Ordinal variable in multiple linear regression model?

I am estimating the price of a car using past data (in other words, Predictive Multiple Linear Regression) and one of the variables is a rating from 1 to 4 of a particular guideline. How do I treat ...
user avatar
0 votes
0 answers
26 views

Help with computing convolution of gaussian and dirac delta

I'm trying to calculate message passing in Trueskill factor, Trueskill paper. Given only two players competing, the message from difference factor to winner team node t1 would be $$ \begin{align} m_{...
user avatar
1 vote
0 answers
21 views

Creating a combined rank score using variables a, b, c, and d, (each is a score 1-100) for constituents using a platform?

I am tasked with creating an overall score that will be used to represent a supplier's holistic value on our site. For example, consumers of supplier's service will want to find suppliers on our site ...
user avatar
  • 11
1 vote
1 answer
26 views

Finding rating for individual based on multiple raters

I am trying to find an individual agent's score based on customer scores (raters) while the customer only rates their overall experience. Context The customer enters a shop and meets a service agent ...
user avatar
  • 41
1 vote
1 answer
14 views

Principled Method for Reweighting Labels Based on Labeler Consistency?

I have a large dataset where each datum has been labeled by multiple raters, and each rater has labeled multiple datums. Rather than assigning to each datum the naive average of all of the labels ...
user avatar
  • 1,679
0 votes
0 answers
14 views

Compare ratings of players over different leagues?

I want to compare ratings of players from different leagues and predict rating of player in a league he/she didn't participate in. Rating of a player is estimated within a league where he was playing. ...
user avatar
0 votes
1 answer
62 views

Is there a statistical measure that combines difference in means and dispersion?

I am working on an academic physics problem, where I am trying to determine favourable variables in two data sets, where both data sets have the same variables. The most favourable variables would ...
user avatar
0 votes
1 answer
254 views

Ordinal Logistic Regression or mixed-model ANOVA?

My experiment is a 2x2x2 mixed design, with one between-subjects factor and the other 2 IVs as within-subjects factors. All variables have two levels. My DV comes in the form of profundity rating ...
user avatar
  • 139
1 vote
1 answer
796 views

Statistical test for preference data through average differences or is other method more suitable?

I need your help. I'm designing a questionary and there I want test which version the people prefere. Which of the following methods is better and which statistical test for significance in each ...
user avatar
1 vote
1 answer
487 views

What are the Elo formulas when assuming performance to be logistically distributed?

I'm struggling to find a definite source on the Elo rating system but from what I understand: it originally assumes players performance to be normally distributed with $\sigma = 1$ the expected ...
user avatar
  • 1,226
1 vote
1 answer
32 views

How to put a collection of random variables in order?

I have a collection of independent Gaussian random variables with unknown means and variances. I want to order them in the order of increasing mean value. For this, I want to come up with a score ...
user avatar
0 votes
1 answer
46 views

How would I calculate value with negative numbers?

I have a dataset of basketball player salaries and an estimation of how many wins they contributed to their teams over the season. So a reasonable thing to do with those numbers would be to divide the ...
user avatar
1 vote
1 answer
20 views

Is Using Differnet Evaluators for Head-to-Head Competing Grants a Fair Process?

We evaluate around 100 grant proposals each year using a team of 15 evaluators who all use the same criteria. 100 grant proposals is a lot for any one evaluator to do and they are experiencing ...
user avatar
1 vote
2 answers
103 views

Developing a simple scoring system for client performance

I was hoping someone can point me in the right direction to develop a simple scoring system or algorithm on clients revenue performance over a period of time. I have over 400 clients with their ...
user avatar
1 vote
0 answers
187 views

What is the best method to rank elements with multiple attributes

This is Kumar, the fire beast monk. The left numbers are the base stats, the green numbers are the stats provided by the equip runes. Depending on your runes you can make your monster more stronger. ...
user avatar
1 vote
0 answers
88 views

Ranking on a 3 point scale

Apologies in advance, my background is in CS rather than statistics, so I may use some terms improperly. I have a data set of a few thousand items scored on a 3 point scale, which my research ...
user avatar
  • 141
0 votes
0 answers
21 views

Build customer ratings from subscription datetime data for a recommendation system

I want to build a recommendation system with only some customer's subscription and unsubscription date. I have a database that looks like: ...
user avatar
2 votes
0 answers
15 views

Should reviews/ratings data be considered as repeated measures?

Suppose I have some ratings/review data for a type of product (say coffee). There are many different kinds of coffee and there are many different subjects reviewing coffee and also some covariates ...
user avatar
  • 1,250
1 vote
0 answers
162 views

How to correlate movie taste?

I am looking for the best known way to correlate movie taste, i.e. to determine which people out of a large pool deviate the least from your own movie scores. One site with this purpose is Criticker. ...
user avatar
1 vote
1 answer
36 views

What model should I use?

I was having a discussion with my father about building an index that ranks cities as a function of multiple variables. For example, ranking n=20 cities based on $$$/SQFT and air quality. I would ...
user avatar
1 vote
1 answer
399 views

How to better rank based on ratings that takes the number of reviews into account?

Normally on most websites when items are being ranked based on reviews, they’re simply based on the average rating (out of 5 or 10); however, I believe an item that has an average rating of 4 based on ...
user avatar
2 votes
0 answers
38 views

What is the name of this test for the probability of differences between lists and is it valid?

And are there any better methods? In a 1969 book about investing, the author Burton P. Fabricand describes a method for assessing the effectiveness of a stock picker. Ask the picker to provide a list ...
user avatar
0 votes
2 answers
86 views

Turning a player's points value into a 1-10 match rating scale? [closed]

So I have the players "fantasy score" stat. This stat is the sum of their ingame stats multiplied by preset weights. These weights are from the video game DOTA 2's official fantasy points weights. For ...
user avatar
  • 3
9 votes
2 answers
249 views

Elo-type ranking system that incorporates game score

The Elo rating system is used to calculate relative skill levels between individuals or teams. It can be applied to many types of games and sports, but when applied, it only considers wins and losses. ...
user avatar
  • 1,042
0 votes
1 answer
359 views

comparing bipolar scale to unipolar scale

I'm trying to build some surveys and I want to ask the following questions: After 3 weeks, how do you feel about decision to join our company? Very Dissatisfied Moderately Dissatisfied Slightly ...
user avatar
  • 209
2 votes
1 answer
115 views

best-worst ranking: do all items have to be blocked together at least 1 time?

Here there are instructions to build Max-diff (or Best-worst) ranking questionnaire: In an ideal world, a MaxDiff experimental design has the following characteristics, where each alternative appears:...
user avatar
3 votes
1 answer
470 views

How to decide the number of options to give to people in a poll? (for opinion based, subjective contexts)

Just to be clear, this question is open ended on purpose, I am fine with(or even prefer to) being pointed to a few research papers, and proper areas related to this subject. I am not sure what I am ...
user avatar
  • 33
0 votes
1 answer
48 views

Averaging bang for the buck

I'm building a table/graph to compare multiple video cards according to a given benchmark. The benchmark consists of multiple measurements using games. For each benchmark measurement, I build a ...
user avatar
  • 101
1 vote
1 answer
100 views

Which rating variable is affecting the main rating?

I have 3 types of rating, the restaurant rating (main rating), food rating, and service rating. Each of these ratings go from 0 to 2. I would like to know if it's possible to tell if a restaurant is ...
user avatar
  • 111
2 votes
0 answers
115 views

Considering sample size when comparing average ratings

Consider, for example, an online shop listing multiple products available for purchase. Each product has an average rating from 1 to 5 calculated from all that product's customer ratings to date. To ...
user avatar
  • 21
7 votes
2 answers
1k views

General principles for extending the Elo system to games in which the margin of victory matters

Note I am a first-time user of this site, and am not sure what determines how broad a question is. I consciously limited the question to general principles in order to avoid making it too broad (by ...
user avatar
3 votes
1 answer
464 views

Does Elo rating system address isolated groups of players?

I am looking to apply a flavor of the Elo Rating System to a game that is played all over the world (not chess) and am toying with the idea of adding what I might consider an "enhancement" to it. So ...
user avatar
  • 143
1 vote
1 answer
74 views

One sample test for helpfulness ratings

I have some questionnaire data were participants rated with [0, 1, 2, 3, 4] whether a product helped with respect to several attributes. 0 indicates that it didn't help at all and 4 is very much. I ...
user avatar
0 votes
1 answer
33 views

Magnifying the outliers in a rating system to rank objects better

1) I am trying to design a product for weekly ratings (1-5) of an entity. The entity's rating will be reset every year. I know most of the weeks the rating will be the same (3) and I want a way to ...
user avatar
2 votes
1 answer
42 views

Are average online ratings of products (say out of 5) assumed to be on a linear scale as understood by the user?

Suppose 2 washing machines A & B are rated as 3.5 and 4.0 (out of 5) respectively, vs another 2, C & D, that are rated as 4.2 and 4.7 respectively. Can I make some useful inference based on ...
user avatar
  • 175
3 votes
1 answer
137 views

How much information is gained when sampling from a random distribution?

This is a problem I have encountered while considering ELO ratings for a game like Go. Suppose we have two players, one has a known rating, the other's is unknown but assumed to be close to the first....
user avatar
2 votes
0 answers
59 views

How can I rate the predictive power of a function through a heatmap?

Context: I use a certain function that is supposed to estimate the possibility of an event "B" to occur in a given area, following the occurence of event "A". The input is a map of the area, and the ...
user avatar
0 votes
0 answers
31 views

Finding an estimate of value from rated examples

Given a sample of items with known monetary values, say 10 cars, and a person rating each car in turn as being more or less valuable than their own, what would be the best way of estimating the value ...
user avatar
1 vote
0 answers
31 views

Determining rating function for severity measure

I have data of a medical experiment and want to run a linear regression. More precisely, I have severity degrees of a disease of two independent groups (treatment and control group). The severity ...
user avatar
  • 111