Skip to main content

Questions tagged [rating]

"A rating is the evaluation or assessment of something, in terms of quality (as with a critic rating a novel), quantity (as with an athlete being rated by his or her statistics), or some combination of both" (Wikipedia).

Filter by
Sorted by
Tagged with
0 votes
0 answers
5 views

What is the formula to calculate win probability based on GLICKO1 and GLICKO2 ratings? [closed]

Given non-draw game, rating, mu, phi and sigma for GLICO1 or GLICO2 systems, how to calculate win probability for parties? I've searched information about these systems, e.g. http://glicko.net/glicko/...
SS_Rebelious's user avatar
1 vote
0 answers
54 views

Modeling player interactions in multi-dimensional rating systems

In traditional rating systems (such as Elo), a player's strength is represented by a single scalar value, which is assumed to be consistent across different opponents. However, in some games, the ...
mb1's user avatar
  • 11
0 votes
0 answers
23 views

UBCF rating predict for binary data in R package recommenderlab

In the R package recommenderlab's predict function for binary data (specifically unary for market basket analysis), why does the code do the following for the weighted UBCF model? ...
Allan Amorim's user avatar
0 votes
0 answers
18 views

Scoring races + ranking index, with Bayesian approach

Challenge: Is this the best approach for scoring multicompetitor races? How do I account for both uncertain prior & uncertain evidence when scoring? Case: athletes getting scores in each race, ...
Daniel Westergren's user avatar
2 votes
0 answers
33 views

Best way of determining how unique an inspector's ratings are compared to the average inspector

Noob question. I have a full population of every inspector rating for all items being rated. Each inspector has inspected a subset of the items and given them a rating, but different inspectors have ...
inspecting_inspectors's user avatar
1 vote
1 answer
166 views

Normalizing a data set by using z-scores from a related data set

I am trying to come up with a normalization for a set of data. My company puts our products through a series of testers (people) to score our products on key metrics. I am able to access all records ...
Citanaf's user avatar
  • 13
1 vote
1 answer
40 views

Should I treat a single response as multiple ones in separate models?

I have this data where participants listened to each stimulus and gave a rating (1~5) for each. But each stimulus contains multiple sentence-like units (i.e., intonational phrase) within it, and for ...
mousemouse's user avatar
0 votes
0 answers
24 views

Elo rating: how many comparisons necessary to achieve accurate rating with n players given no prior knowledge?

If I have n participants in a two-player-game league, whose initial ratings are unknown, is there a good heuristic for how many games/comparisons/pair-faceoffs are required to achieve an accurate ...
2080's user avatar
  • 101
7 votes
3 answers
1k views

On the cradle of the Elo Rating System, and how to find it

A definitive proposal can be found in: Elo, Arpad E. (August, 1967). The Proposed USCF Rating System, Its Development, Theory, and Applications. Chess Life XXII (8): 242-247. How exactly are THE ...
clp's user avatar
  • 308
2 votes
0 answers
41 views

Dynamic Time Warping with polarity?

I am trying to quantify the cumulative difference between two time series using dynamic time warping. For example, here I calculated and plotted the DTW distance between Speaker rating (pink line) and ...
Dasha's user avatar
  • 21
1 vote
0 answers
63 views

A Rating Formula that Combines Two Metrics

If I have two metrics, - number of reviews and the rating out of 5, and I want to combine them into a single meaningful rating that can rank a list, how can I do that? For example, let's say I have ...
Simon's user avatar
  • 11
3 votes
1 answer
21 views

What are some techniques to calculate the highsest average when the number of data points is important?

A very simple example here would be restaurants with online reviews. On a platform where every resturant might have a different number of reviews with a vote ranging from 1 to 5, which restaurant is ...
heapOverflow's user avatar
2 votes
1 answer
55 views

For human annotation projects, what are some commonly used metrics to assess grader reliability?

Lots of machine learning datasets are now created by having human raters annotate and provide labels to questions. Usually, a gold set is the most robust way of seeing if the raters are doing a good ...
user321627's user avatar
  • 4,342
3 votes
1 answer
299 views

Skill score/Rating systems that take into consideration margin of victory

I am trying to create ratings/skill scores using ranking systems like Elo, TrueSkill, and OpenSkill. All of these systems are similar in that they have a distribution that represents players' skill ...
GBPU's user avatar
  • 173
1 vote
1 answer
64 views

Supra-additive relations among timeseries (separate ratings of a single stimulus)

I conducted an online behavioural study (N=20) where people listened to a song with lyrics, and rated (in three counter-balanced conditions) the tension they perceived in the music alone (M), the ...
z8080's user avatar
  • 2,382
0 votes
1 answer
76 views

What is the points valuation for this problem (assuming that $\!5\!-\!$pointer worth $1\!$ point is standard)

Given 4 teams in a tournament, they versus against all the other teams once for each to reach best-possible place. Three points are awarded to the team winning a match, with no points awarded to the ...
user avatar
1 vote
1 answer
37 views

How was "above chance" defined in this paper and confidence intervals calculated? [Rating method: Best-worst scaling]

I was reading a paper about ratings (Hollis & Westbury, 2018), precisely, best worst scaling, where in each trial out of N items (in this case N = 4) the best and the worst item have to be ...
Sahila's user avatar
  • 13
0 votes
0 answers
663 views

Computing ICCs in Matlab, to assess rater consistency (inter-rater agreement)

In an experiment, participants rated each of several musical stimuli on 5 aspects (ratings A to E), across ~3,000 trials. Stimuli were performed (by a previous set of subjects) either solo or duo (...
z8080's user avatar
  • 2,382
0 votes
0 answers
234 views

Interaction between variables to create a ranking system

I was wondering if there is a standard way to rank cases by multiple variables. When we have only 1 quantitative variable, the ranking is trivial, you sort the cases by comparing the magnitude between ...
gabriel's user avatar
  • 93
6 votes
1 answer
435 views

help with formula to calculate Bayesian ranking of M-star reviews

I wonder, if someone could help to understand a formula from a book please. Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference (Addison-Wesley Data & Analytics) (...
cs0815's user avatar
  • 2,245
2 votes
2 answers
421 views

Why are rating distributions 'smoother' when there are more players?

Consider the following rating distributions by variant/time control on lichess. The bullet, blitz and rapid rating distributions look very, well, 'smooth' (or holomorphic/analytic or harmonic or ...
BCLC's user avatar
  • 2,444
0 votes
0 answers
60 views

Is it appropriate to use difference scores in this context?

I am trying to compare the level of rating accuracy of two groups of participants (say Sample A and B). The study design is as is: I asked a consumer panel to rate how much they liked three products. ...
Theo's user avatar
  • 1
3 votes
1 answer
170 views

Prove or disprove that rating worth in online chess depends on the time of the day

Crosspost from Chess.SE. TL;DR I feel that ratings on a chess website are worth differently depending on the time of the day, and I want to validate the claim. I have a complete history of games ...
Ivan Smirnov's user avatar
1 vote
2 answers
282 views

Comparing ratings & statistical difference

I have a dataset where I'm comparing overall ratings between multiple "products." It looks something like this: Product 1: 3.5 Product 2: 4.1 Product 3: 3.7 ... Product 7: 3.3 What I'm doing ...
DD1229's user avatar
  • 111
0 votes
1 answer
146 views

Q: Aggregating audience and critic movie ratings, and creating a composite rating of the two

I’m building a movie review aggregation site that combines user ratings and critical reviews for a given movie. The objective is to create a list of the “best of the best” movies which were rated ...
GaussianBlur's user avatar
1 vote
1 answer
145 views

Weighted Kappa with rating variables that have different set of categories

I want to measure the inter-agreement rate between two physicians that rated the same set of images with two different methods (rater1 used method1, rater2 used method2) BUT Rater1 rated the the ...
learners's user avatar
  • 579
1 vote
1 answer
6k views

Ordinal variable in multiple linear regression model?

I am estimating the price of a car using past data (in other words, Predictive Multiple Linear Regression) and one of the variables is a rating from 1 to 4 of a particular guideline. How do I treat ...
reasonStore's user avatar
0 votes
0 answers
63 views

Help with computing convolution of gaussian and dirac delta

I'm trying to calculate message passing in Trueskill factor, Trueskill paper. Given only two players competing, the message from difference factor to winner team node t1 would be $$ \begin{align} m_{...
권영주's user avatar
1 vote
0 answers
24 views

Creating a combined rank score using variables a, b, c, and d, (each is a score 1-100) for constituents using a platform?

I am tasked with creating an overall score that will be used to represent a supplier's holistic value on our site. For example, consumers of supplier's service will want to find suppliers on our site ...
cofo793's user avatar
  • 11
1 vote
1 answer
31 views

Finding rating for individual based on multiple raters

I am trying to find an individual agent's score based on customer scores (raters) while the customer only rates their overall experience. Context The customer enters a shop and meets a service agent ...
Ctrl's user avatar
  • 41
2 votes
1 answer
22 views

Principled Method for Reweighting Labels Based on Labeler Consistency?

I have a large dataset where each datum has been labeled by multiple raters, and each rater has labeled multiple datums. Rather than assigning to each datum the naive average of all of the labels ...
jon_simon's user avatar
  • 2,049
0 votes
0 answers
18 views

Compare ratings of players over different leagues?

I want to compare ratings of players from different leagues and predict rating of player in a league he/she didn't participate in. Rating of a player is estimated within a league where he was playing. ...
Kajetan Poliak's user avatar
1 vote
1 answer
100 views

Is there a statistical measure that combines difference in means and dispersion?

I am working on an academic physics problem, where I am trying to determine favourable variables in two data sets, where both data sets have the same variables. The most favourable variables would ...
Cairan Van Rooyen's user avatar
0 votes
1 answer
529 views

Ordinal Logistic Regression or mixed-model ANOVA?

My experiment is a 2x2x2 mixed design, with one between-subjects factor and the other 2 IVs as within-subjects factors. All variables have two levels. My DV comes in the form of profundity rating ...
NickB's user avatar
  • 139
1 vote
1 answer
1k views

Statistical test for preference data through average differences or is other method more suitable?

I need your help. I'm designing a questionary and there I want test which version the people prefere. Which of the following methods is better and which statistical test for significance in each ...
qwer1993's user avatar
3 votes
1 answer
916 views

What are the Elo formulas when assuming performance to be logistically distributed?

I'm struggling to find a definite source on the Elo rating system but from what I understand: it originally assumes players performance to be normally distributed with $\sigma = 1$ the expected ...
user2740's user avatar
  • 1,376
1 vote
1 answer
37 views

How to put a collection of random variables in order?

I have a collection of independent Gaussian random variables with unknown means and variances. I want to order them in the order of increasing mean value. For this, I want to come up with a score ...
Volodymyr Kruglov's user avatar
0 votes
1 answer
49 views

How would I calculate value with negative numbers?

I have a dataset of basketball player salaries and an estimation of how many wins they contributed to their teams over the season. So a reasonable thing to do with those numbers would be to divide the ...
Clayton Johnson's user avatar
1 vote
1 answer
23 views

Is Using Differnet Evaluators for Head-to-Head Competing Grants a Fair Process?

We evaluate around 100 grant proposals each year using a team of 15 evaluators who all use the same criteria. 100 grant proposals is a lot for any one evaluator to do and they are experiencing ...
Adam Cole's user avatar
1 vote
2 answers
128 views

Developing a simple scoring system for client performance

I was hoping someone can point me in the right direction to develop a simple scoring system or algorithm on clients revenue performance over a period of time. I have over 400 clients with their ...
Daniel James Canil's user avatar
1 vote
0 answers
508 views

What is the best method to rank elements with multiple attributes

This is Kumar, the fire beast monk. The left numbers are the base stats, the green numbers are the stats provided by the equip runes. Depending on your runes you can make your monster more stronger. ...
Juan Carlos Oropeza's user avatar
1 vote
0 answers
150 views

Ranking on a 3 point scale

Apologies in advance, my background is in CS rather than statistics, so I may use some terms improperly. I have a data set of a few thousand items scored on a 3 point scale, which my research ...
knite's user avatar
  • 141
0 votes
0 answers
22 views

Build customer ratings from subscription datetime data for a recommendation system

I want to build a recommendation system with only some customer's subscription and unsubscription date. I have a database that looks like: ...
Boubacar Traoré's user avatar
2 votes
0 answers
21 views

Should reviews/ratings data be considered as repeated measures?

Suppose I have some ratings/review data for a type of product (say coffee). There are many different kinds of coffee and there are many different subjects reviewing coffee and also some covariates ...
Marcel's user avatar
  • 1,380
1 vote
0 answers
291 views

How to correlate movie taste?

I am looking for the best known way to correlate movie taste, i.e. to determine which people out of a large pool deviate the least from your own movie scores. One site with this purpose is Criticker. ...
summerrain's user avatar
1 vote
1 answer
44 views

What model should I use?

I was having a discussion with my father about building an index that ranks cities as a function of multiple variables. For example, ranking n=20 cities based on $$$/SQFT and air quality. I would ...
Blake Shurtz's user avatar
2 votes
1 answer
1k views

How to better rank based on ratings that takes the number of reviews into account?

Normally on most websites when items are being ranked based on reviews, they’re simply based on the average rating (out of 5 or 10); however, I believe an item that has an average rating of 4 based on ...
ramtin alami's user avatar
2 votes
0 answers
40 views

What is the name of this test for the probability of differences between lists and is it valid?

And are there any better methods? In a 1969 book about investing, the author Burton P. Fabricand describes a method for assessing the effectiveness of a stock picker. Ask the picker to provide a list ...
HumbleOrange's user avatar
0 votes
2 answers
180 views

Turning a player's points value into a 1-10 match rating scale? [closed]

So I have the players "fantasy score" stat. This stat is the sum of their ingame stats multiplied by preset weights. These weights are from the video game DOTA 2's official fantasy points weights. For ...
Gorlan's user avatar
  • 3
10 votes
2 answers
1k views

Elo-type ranking system that incorporates game score

The Elo rating system is used to calculate relative skill levels between individuals or teams. It can be applied to many types of games and sports, but when applied, it only considers wins and losses. ...
Figaro's user avatar
  • 1,152