Questions tagged [rating]
"A rating is the evaluation or assessment of something, in terms of quality (as with a critic rating a novel), quantity (as with an athlete being rated by his or her statistics), or some combination of both" (Wikipedia).
146
questions
1
vote
0
answers
52
views
Modeling player interactions in multi-dimensional rating systems
In traditional rating systems (such as Elo), a player's strength is represented by a single scalar value, which is assumed to be consistent across different opponents. However, in some games, the ...
- 11
0
votes
0
answers
14
views
UBCF rating predict for binary data in R package recommenderlab
In the R package recommenderlab's predict function for binary data (specifically unary for market basket analysis), why does the code do the following for the weighted UBCF model?
...
0
votes
0
answers
15
views
Scoring races + ranking index, with Bayesian approach
Challenge: Is this the best approach for scoring multicompetitor races? How do I account for both uncertain prior & uncertain evidence when scoring?
Case: athletes getting scores in each race, ...
2
votes
0
answers
29
views
Best way of determining how unique an inspector's ratings are compared to the average inspector
Noob question.
I have a full population of every inspector rating for all items being rated. Each inspector has inspected a subset of the items and given them a rating, but different inspectors have ...
1
vote
1
answer
79
views
Normalizing a data set by using z-scores from a related data set
I am trying to come up with a normalization for a set of data. My company puts our products through a series of testers (people) to score our products on key metrics. I am able to access all records ...
- 13
1
vote
1
answer
38
views
Should I treat a single response as multiple ones in separate models?
I have this data where participants listened to each stimulus and gave a rating (1~5) for each. But each stimulus contains multiple sentence-like units (i.e., intonational phrase) within it, and for ...
- 11
0
votes
0
answers
16
views
Elo rating: how many comparisons necessary to achieve accurate rating with n players given no prior knowledge?
If I have n participants in a two-player-game league, whose initial ratings are unknown, is there a good heuristic for how many games/comparisons/pair-faceoffs are required to achieve an accurate ...
- 101
6
votes
3
answers
1k
views
On the cradle of the Elo Rating System, and how to find it
A definitive proposal can be found in:
Elo, Arpad E. (August, 1967).
The Proposed USCF Rating System,
Its Development, Theory, and Applications.
Chess Life XXII (8): 242-247.
How exactly are
THE ...
- 288
1
vote
0
answers
26
views
Dynamic Time Warping with polarity?
I am trying to quantify the cumulative difference between two time series using dynamic time warping. For example, here I calculated and plotted the DTW distance between Speaker rating (pink line) and ...
- 11
1
vote
0
answers
44
views
A Rating Formula that Combines Two Metrics
If I have two metrics, - number of reviews and the rating out of 5, and I want to combine them into a single meaningful rating that can rank a list, how can I do that?
For example, let's say I have ...
- 11
3
votes
1
answer
21
views
What are some techniques to calculate the highsest average when the number of data points is important?
A very simple example here would be restaurants with online reviews. On a platform where every resturant might have a different number of reviews with a vote ranging from 1 to 5, which restaurant is ...
- 131
2
votes
1
answer
45
views
For human annotation projects, what are some commonly used metrics to assess grader reliability?
Lots of machine learning datasets are now created by having human raters annotate and provide labels to questions. Usually, a gold set is the most robust way of seeing if the raters are doing a good ...
- 3,922
2
votes
1
answer
194
views
Skill score/Rating systems that take into consideration margin of victory
I am trying to create ratings/skill scores using ranking systems like Elo, TrueSkill, and OpenSkill. All of these systems are similar in that they have a distribution that represents players' skill ...
- 143
1
vote
1
answer
64
views
Supra-additive relations among timeseries (separate ratings of a single stimulus)
I conducted an online behavioural study (N=20) where people listened to a song with lyrics, and rated (in three counter-balanced conditions) the tension they perceived in the music alone (M), the ...
- 2,120
0
votes
1
answer
75
views
What is the points valuation for this problem (assuming that $\!5\!-\!$pointer worth $1\!$ point is standard)
Given 4 teams in a tournament, they versus against all the other teams once for each to reach best-possible place. Three points are awarded to the team winning a match, with no points awarded to the ...
user354475
1
vote
1
answer
34
views
How was "above chance" defined in this paper and confidence intervals calculated? [Rating method: Best-worst scaling]
I was reading a paper about ratings (Hollis & Westbury, 2018), precisely, best worst scaling, where in each trial out of N items (in this case N = 4) the best and the worst item have to be ...
- 13
0
votes
0
answers
564
views
Computing ICCs in Matlab, to assess rater consistency (inter-rater agreement)
In an experiment, participants rated each of several musical stimuli on 5 aspects (ratings A to E), across ~3,000 trials. Stimuli were performed (by a previous set of subjects) either solo or duo (...
- 2,120
0
votes
0
answers
192
views
Interaction between variables to create a ranking system
I was wondering if there is a standard way to rank cases by multiple variables.
When we have only 1 quantitative variable, the ranking is trivial, you sort the cases by comparing the magnitude between ...
- 93
5
votes
1
answer
410
views
help with formula to calculate Bayesian ranking of M-star reviews
I wonder, if someone could help to understand a formula from a book please.
Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference (Addison-Wesley Data & Analytics) (...
- 1,987
2
votes
2
answers
388
views
Why are rating distributions 'smoother' when there are more players?
Consider the following rating distributions by variant/time control on lichess.
The bullet, blitz and rapid rating distributions look very, well, 'smooth' (or holomorphic/analytic or harmonic or ...
- 2,394
0
votes
0
answers
52
views
Is it appropriate to use difference scores in this context?
I am trying to compare the level of rating accuracy of two groups of participants (say Sample A and B). The study design is as is: I asked a consumer panel to rate how much they liked three products. ...
- 1
3
votes
1
answer
165
views
Prove or disprove that rating worth in online chess depends on the time of the day
Crosspost from Chess.SE.
TL;DR
I feel that ratings on a chess website are worth differently depending on the time of the day, and I want to validate the claim. I have a complete history of games ...
- 131
1
vote
2
answers
213
views
Comparing ratings & statistical difference
I have a dataset where I'm comparing overall ratings between multiple "products." It looks something like this:
Product 1: 3.5
Product 2: 4.1
Product 3: 3.7
...
Product 7: 3.3
What I'm doing ...
- 111
0
votes
1
answer
122
views
Q: Aggregating audience and critic movie ratings, and creating a composite rating of the two
I’m building a movie review aggregation site that combines user ratings and critical reviews for a given movie. The objective is to create a list of the “best of the best” movies which were rated ...
1
vote
1
answer
133
views
Weighted Kappa with rating variables that have different set of categories
I want to measure the inter-agreement rate between two physicians that rated the same set of images with two different methods (rater1 used method1, rater2 used method2) BUT
Rater1 rated the the ...
- 569
1
vote
1
answer
5k
views
Ordinal variable in multiple linear regression model?
I am estimating the price of a car using past data (in other words, Predictive Multiple Linear Regression) and one of the variables is a rating from 1 to 4 of a particular guideline. How do I treat ...
- 23
0
votes
0
answers
41
views
Help with computing convolution of gaussian and dirac delta
I'm trying to calculate message passing in Trueskill factor, Trueskill paper. Given only two players competing, the message from difference factor to winner team node t1 would be
$$
\begin{align}
m_{...
- 16
1
vote
0
answers
24
views
Creating a combined rank score using variables a, b, c, and d, (each is a score 1-100) for constituents using a platform?
I am tasked with creating an overall score that will be used to represent a supplier's holistic value on our site. For example, consumers of supplier's service will want to find suppliers on our site ...
- 11
1
vote
1
answer
30
views
Finding rating for individual based on multiple raters
I am trying to find an individual agent's score based on customer scores (raters) while the customer only rates their overall experience.
Context
The customer enters a shop and meets a service agent ...
- 41
2
votes
1
answer
22
views
Principled Method for Reweighting Labels Based on Labeler Consistency?
I have a large dataset where each datum has been labeled by multiple raters, and each rater has labeled multiple datums. Rather than assigning to each datum the naive average of all of the labels ...
- 1,919
0
votes
0
answers
18
views
Compare ratings of players over different leagues?
I want to compare ratings of players from different leagues and predict rating of player in a league he/she didn't participate in. Rating of a player is estimated within a league where he was playing.
...
1
vote
1
answer
96
views
Is there a statistical measure that combines difference in means and dispersion?
I am working on an academic physics problem, where I am trying to determine favourable variables in two data sets, where both data sets have the same variables.
The most favourable variables would ...
0
votes
1
answer
484
views
Ordinal Logistic Regression or mixed-model ANOVA?
My experiment is a 2x2x2 mixed design, with one between-subjects factor and the other 2 IVs as within-subjects factors. All variables have two levels.
My DV comes in the form of profundity rating ...
- 129
1
vote
1
answer
1k
views
Statistical test for preference data through average differences or is other method more suitable?
I need your help. I'm designing a questionary and there I want test which version the people prefere. Which of the following methods is better and which statistical test for significance in each ...
- 25
3
votes
1
answer
854
views
What are the Elo formulas when assuming performance to be logistically distributed?
I'm struggling to find a definite source on the Elo rating system but from what I understand:
it originally assumes players performance to be normally distributed with $\sigma = 1$
the expected ...
- 1,366
1
vote
1
answer
36
views
How to put a collection of random variables in order?
I have a collection of independent Gaussian random variables with unknown means and variances. I want to order them in the order of increasing mean value. For this, I want to come up with a score ...
0
votes
1
answer
49
views
How would I calculate value with negative numbers?
I have a dataset of basketball player salaries and an estimation of how many wins they contributed to their teams over the season. So a reasonable thing to do with those numbers would be to divide the ...
1
vote
1
answer
21
views
Is Using Differnet Evaluators for Head-to-Head Competing Grants a Fair Process?
We evaluate around 100 grant proposals each year using a team of 15 evaluators who all use the same criteria. 100 grant proposals is a lot for any one evaluator to do and they are experiencing ...
- 11
1
vote
2
answers
125
views
Developing a simple scoring system for client performance
I was hoping someone can point me in the right direction to develop a simple scoring system or algorithm on clients revenue performance over a period of time.
I have over 400 clients with their ...
1
vote
0
answers
457
views
What is the best method to rank elements with multiple attributes
This is Kumar, the fire beast monk. The left numbers are the base stats, the green numbers are the stats provided by the equip runes. Depending on your runes you can make your monster more stronger.
...
1
vote
0
answers
133
views
Ranking on a 3 point scale
Apologies in advance, my background is in CS rather than statistics, so I may use some terms improperly.
I have a data set of a few thousand items scored on a 3 point scale, which my research ...
- 141
0
votes
0
answers
22
views
Build customer ratings from subscription datetime data for a recommendation system
I want to build a recommendation system with only some customer's subscription and unsubscription date. I have a database that looks like:
...
- 101
2
votes
0
answers
21
views
Should reviews/ratings data be considered as repeated measures?
Suppose I have some ratings/review data for a type of product (say coffee). There are many different kinds of coffee and there are many different subjects reviewing coffee and also some covariates ...
- 1,350
1
vote
0
answers
273
views
How to correlate movie taste?
I am looking for the best known way to correlate movie taste, i.e. to determine which people out of a large pool deviate the least from your own movie scores.
One site with this purpose is Criticker. ...
- 111
1
vote
1
answer
44
views
What model should I use?
I was having a discussion with my father about building an index that ranks cities as a function of multiple variables.
For example, ranking n=20 cities based on $$$/SQFT and air quality.
I would ...
- 462
2
votes
1
answer
973
views
How to better rank based on ratings that takes the number of reviews into account?
Normally on most websites when items are being ranked based on reviews, they’re simply based on the average rating (out of 5 or 10); however, I believe an item that has an average rating of 4 based on ...
- 53
2
votes
0
answers
39
views
What is the name of this test for the probability of differences between lists and is it valid?
And are there any better methods?
In a 1969 book about investing, the author Burton P. Fabricand describes a method for assessing the effectiveness of a stock picker. Ask the picker to provide a list ...
- 459
0
votes
2
answers
153
views
Turning a player's points value into a 1-10 match rating scale? [closed]
So I have the players "fantasy score" stat. This stat is the sum of their ingame stats multiplied by preset weights. These weights are from the video game DOTA 2's official fantasy points weights. For ...
- 3
9
votes
2
answers
803
views
Elo-type ranking system that incorporates game score
The Elo rating system is used to calculate relative skill levels between individuals or teams. It can be applied to many types of games and sports, but when applied, it only considers wins and losses. ...
- 1,142
0
votes
1
answer
456
views
comparing bipolar scale to unipolar scale
I'm trying to build some surveys and I want to ask the following questions:
After 3 weeks, how do you feel about decision to join our company?
Very Dissatisfied
Moderately Dissatisfied
Slightly ...
- 219