Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [rating]

"A rating is the evaluation or assessment of something, in terms of quality (as with a critic rating a novel), quantity (as with an athlete being rated by his or her statistics), or some combination of both" (Wikipedia).

1
vote
1answer
18 views

Is Using Differnet Evaluators for Head-to-Head Competing Grants a Fair Process?

We evaluate around 100 grant proposals each year using a team of 15 evaluators who all use the same criteria. 100 grant proposals is a lot for any one evaluator to do and they are experiencing ...
0
votes
0answers
18 views

Model reputation with beta distribution

I am trying to develop a reputation score in which the reputation is calculated by the ratings that buyers give to sellers. What i am trying is to replicate a paper very well known http://faculty.neu....
1
vote
2answers
42 views

Developing a simple scoring system for client performance

I was hoping someone can point me in the right direction to develop a simple scoring system or algorithm on clients revenue performance over a period of time. I have over 400 clients with their ...
1
vote
0answers
17 views

What is the best method to rank elements with multiple attributes

This is Kumar, the fire beast monk. The left numbers are the base stats, the green numbers are the stats provided by the equip runes. Depending on your runes you can make your monster more stronger. ...
0
votes
0answers
58 views

K factor for Elo Rating System

The Elo Rating System can be used to rate players (or entities in general) and predict the outcomes of games or any 2-party events. It puts players/parties into categories according to their ratings ...
1
vote
0answers
25 views

Ranking on a 3 point scale

Apologies in advance, my background is in CS rather than statistics, so I may use some terms improperly. I have a data set of a few thousand items scored on a 3 point scale, which my research ...
0
votes
0answers
19 views

Build customer ratings from subscription datetime data for a recommendation system

I want to build a recommendation system with only some customer's subscription and unsubscription date. I have a database that looks like: ...
0
votes
0answers
33 views

Correlation of two values with opposite rank/rating

Consider the following set: ...
0
votes
0answers
39 views

Sorting By User Ratings - Overlapping Wilson Confidence Intervals

I'm working on a personal project to rank user ratings on a 5 star scale and have implemented Evan Miller's often-referenced Wilson Score Interval calculation with the help of this comment. My data ...
1
vote
0answers
10 views

Should reviews/ratings data be considered as repeated measures?

Suppose I have some ratings/review data for a type of product (say coffee). There are many different kinds of coffee and there are many different subjects reviewing coffee and also some covariates ...
1
vote
0answers
58 views

How to correlate movie taste?

I am looking for the best known way to correlate movie taste, i.e. to determine which people out of a large pool deviate the least from your own movie scores. One site with this purpose is Criticker. ...
0
votes
1answer
30 views

What model should I use?

I was having a discussion with my father about building an index that ranks cities as a function of multiple variables. For example, ranking n=20 cities based on $$$/SQFT and air quality. I would ...
0
votes
0answers
36 views

Normalizing differing rating styles of users (recommender system data)

In user provided ratings (5-star scale) for a recommender system, users' varying rating styles are quite noticeable. For instance, some users like to rate either 1 or 5 (dislike / like). Others are ...
0
votes
1answer
34 views

How to better rank based on ratings that takes the number of reviews into account?

Normally on most websites when items are being ranked based on reviews, they’re simply based on the average rating (out of 5 or 10); however, I believe an item that has an average rating of 4 based on ...
2
votes
0answers
32 views

What is the name of this test for the probability of differences between lists and is it valid?

And are there any better methods? In a 1969 book about investing, the author Burton P. Fabricand describes a method for assessing the effectiveness of a stock picker. Ask the picker to provide a list ...
0
votes
2answers
29 views

Turning a player's points value into a 1-10 match rating scale? [closed]

So I have the players "fantasy score" stat. This stat is the sum of their ingame stats multiplied by preset weights. These weights are from the video game DOTA 2's official fantasy points weights. For ...
4
votes
0answers
61 views

Elo-type ranking system that incorporates game score

The Elo rating system is used to calculate relative skill levels between individuals or teams. It can be applied to many types of games and sports, but when applied, it only considers wins and losses. ...
0
votes
0answers
34 views

Score to assess ranking of predicted outcomes

Is there any quantitative measure to evaluate a predictive model's ability rank units correctly. For example, I am trying to predict a continuous outcome. The error of my predicted versus actual value ...
0
votes
1answer
83 views

comparing bipolar scale to unipolar scale

I'm trying to build some surveys and I want to ask the following questions: After 3 weeks, how do you feel about decision to join our company? Very Dissatisfied Moderately Dissatisfied ...
3
votes
1answer
71 views

best-worst ranking: do all items have to be blocked together at least 1 time?

Here there are instructions to build Max-diff (or Best-worst) ranking questionnaire: In an ideal world, a MaxDiff experimental design has the following characteristics, where each alternative ...
2
votes
1answer
40 views

How to decide the number of options to give to people in a poll? (for opinion based, subjective contexts)

Just to be clear, this question is open ended on purpose, I am fine with(or even prefer to) being pointed to a few research papers, and proper areas related to this subject. I am not sure what I am ...
0
votes
1answer
42 views

Averaging bang for the buck

I'm building a table/graph to compare multiple video cards according to a given benchmark. The benchmark consists of multiple measurements using games. For each benchmark measurement, I build a ...
1
vote
1answer
51 views

Which rating variable is affecting the main rating?

I have 3 types of rating, the restaurant rating (main rating), food rating, and service rating. Each of these ratings go from 0 to 2. I would like to know if it's possible to tell if a restaurant is ...
1
vote
0answers
35 views

Considering sample size when comparing average ratings

Consider, for example, an online shop listing multiple products available for purchase. Each product has an average rating from 1 to 5 calculated from all that product's customer ratings to date. To ...
7
votes
2answers
304 views

General principles for extending the Elo system to games in which the margin of victory matters

Note I am a first-time user of this site, and am not sure what determines how broad a question is. I consciously limited the question to general principles in order to avoid making it too broad (by ...
3
votes
1answer
129 views

Does Elo rating system address isolated groups of players?

I am looking to apply a flavor of the Elo Rating System to a game that is played all over the world (not chess) and am toying with the idea of adding what I might consider an "enhancement" to it. So ...
1
vote
1answer
35 views

One sample test for helpfulness ratings

I have some questionnaire data were participants rated with [0, 1, 2, 3, 4] whether a product helped with respect to several attributes. 0 indicates that it didn't help at all and 4 is very much. I ...
0
votes
1answer
28 views

Magnifying the outliers in a rating system to rank objects better

1) I am trying to design a product for weekly ratings (1-5) of an entity. The entity's rating will be reset every year. I know most of the weeks the rating will be the same (3) and I want a way to ...
2
votes
1answer
36 views

Are average online ratings of products (say out of 5) assumed to be on a linear scale as understood by the user?

Suppose 2 washing machines A & B are rated as 3.5 and 4.0 (out of 5) respectively, vs another 2, C & D, that are rated as 4.2 and 4.7 respectively. Can I make some useful inference based on ...
2
votes
1answer
99 views

How much information is gained when sampling from a random distribution?

This is a problem I have encountered while considering ELO ratings for a game like Go. Suppose we have two players, one has a known rating, the other's is unknown but assumed to be close to the first....
2
votes
0answers
49 views

How can I rate the predictive power of a function through a heatmap?

Context: I use a certain function that is supposed to estimate the possibility of an event "B" to occur in a given area, following the occurence of event "A". The input is a map of the area, and the ...
0
votes
0answers
31 views

Finding an estimate of value from rated examples

Given a sample of items with known monetary values, say 10 cars, and a person rating each car in turn as being more or less valuable than their own, what would be the best way of estimating the value ...
1
vote
0answers
28 views

Determining rating function for severity measure

I have data of a medical experiment and want to run a linear regression. More precisely, I have severity degrees of a disease of two independent groups (treatment and control group). The severity ...
0
votes
1answer
3k views

How to normalize rating in scale of 1 to 5?

Can anyone please tell me How to normalize rating in a scale of 1 to 5? In Yahoo! Movies dataset user has given a rating to a movie on the scale of 1 to 13 and 0 means there is no rating to that ...
0
votes
0answers
37 views

Ranking, rating things based on a set of metrics

I need to rank a set of things belonging to different kind (models) based on a list of performance metrics. For thing of each model, I have the aggregated value ( mean) of let's say 4 metrics and ...
2
votes
0answers
829 views

Recommender Systems with Implicit Feedback Data - Modelling and Updating

I have a very large dataset of user-item interactions. It tells me how many times a user bought, viewed or liked an item - all these actions are in binary form and I don't have any explicit ratings. ...
0
votes
1answer
44 views

Attributing difference in a DV (preference rating) to between-subjects manipulation vs preference differences

Say we want to test which of two products A and B is more liked by the general public, by obtaining ratings from random participants assigned to groups A and B, i.e. a between-subjects design. Let's ...
3
votes
2answers
122 views

Error bars (or their likeness) for rankings

This is a bit of a simple question I would suspect for many users here, but for someone like me who has never covered this in my statistics classes, this is posing a bit of a problem. I want to ...
-1
votes
1answer
102 views

Choosing method for multiple linear regression for 1 to 10 ratings

I'm performing a survey where people will rate some things from 1 to 10 based on their characteristics and I'm having trouble understanding which model to use and how to compile the data. For ...
0
votes
1answer
34 views

Creating a scoring model when different variables have higher or lower as better

I created a scoring model based on 4 metrics. Currently I have them weighted as such: 50%, 20%, 20% and 10%. After calculating and ranking I realized this may not work. The issue I'm facing is that ...
1
vote
1answer
29 views

Maximize expected score of a ranking

I am trying to figure out how to answer 1.1.c from here http://www.cc.gatech.edu/sites/default/files/images/200808_cse_qualifier.pdf. The problem setup is as follows We are then asked to show An ...
1
vote
0answers
114 views

Calculating a composite index in PCA using several principal components [duplicate]

I have a question I was hoping this group could help answer or provide more insight on. I'm looking to construct an overall composite index for counties. Really a simplified version of the County ...
0
votes
1answer
24 views

Rank correlations with very coarse reference

I have some data about 50 hotels (for this example). They are rated 1-5 stars -- the ground truth. Then I have a number of different estimators which use some features to output a score. For example ...
2
votes
1answer
35 views

How do I derive a rating from two variables containing zeros?

I have a dataset of the performance of probands in an experiment. An observer watching the performance noted positive and negative performance remarks on a simple tally, and ended up with something ...
1
vote
0answers
34 views

Learning ranking model leveraging multiple noisy pairwise constraints

I am trying to come up with a probabilistic model that (1) learns a pairwise ranking function (2) and leverage multiple noisy pairwise constraints. My problem setup has two parts: (1) There are $N$ ...
5
votes
1answer
119 views
0
votes
0answers
30 views

Correcting for individual differences in the usage (employed range) of rating scales

Experiments that collect subjective ratings (e.g. on a Likert 1-7 scale) necessarily have to account for between-subjects differences in how the range of the rating scale is used by different people, ...
3
votes
0answers
174 views

Does it make sense to add the principal components together to produce a single index? [duplicate]

I want to build an index from, say, 13 variables. I run a PCA for these 13 variables to produce 13 principal components, 5 of which have an Eigenvalue of more than 1. While some researchers use only ...
0
votes
1answer
242 views

Fleiss' Kappa alternative for ranking

I've a dataset with items for which some users have given a score from 1 to 6. I wanted to compute the Fleiss' kappa to get the inter user agreement. The thing is that a I want a weighted version of ...