Tagged Questions

The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
1answer
16 views

Compare two items when rated by ordinal variables

This question was spurred by one of those tech support sites that asks at the bottom "rate how helpful this article was" and allows you to rate the article 5 for very helpful and 1 for not helpful at ...
0
votes
0answers
34 views

Performance metric

Let's say I have two people, A and B, who are each performing a different task. They have an aptitude score which designates how well they perform any task; A has apitude score a and B has aptitude ...
1
vote
0answers
317 views

Transforming scores between different rating scales – Importance of response options

I'm working on a meta analysis of the mean of a single variable (using arithmetic mean as the effect size, rather than the relationship between two variables). The attitude I'm assessing has been ...
0
votes
0answers
29 views

Compare distributions of user ratings?

I have a bunch of user ratings, over time, for different products, and I'd like to figure out the best ways to analyse and compare them. Here's how the data looks: ...
0
votes
0answers
57 views

How well do analyst ratings predict stock perfromance?

Consider a huge dataset of which the following is a characteristic sample: ...
0
votes
0answers
15 views

Analysing a rating scale with many predictor variables

My response variable is derived from a question in which respondents expressed a preference score (from 1 (do not like) to 10 (like alot)) for a range of scenarios. I would like to analyse this data ...
1
vote
0answers
148 views

Confidence intervals for 6-point Likert scale

I was asked to compare certain ratings between two different groups. The distribution of ratings looks something like this: ...
0
votes
0answers
9 views

When measuring attitudes, does rank-ordering choices have any advantages over rating choices?

I can see an advantage for rating in that you potentially end up with interval data rather than ordinal data. However, I'm struggling to think of what potential advantages there might be to ...
0
votes
0answers
44 views

Multiple Variable Ranking

Before i start, i should mention i am pursuing my under-grad degree, and hence i have little/basic knowledge about machine learning and statistics. I've been meaning to make an application which ...
0
votes
1answer
45 views

How to apply Laplace rule of succession for any possible value in a range?

I am looking to have People rate certain items on a scale of 0-10 based on those items perceived utility to the particular Person performing the rating. Fractional values are permitted. Giving no ...
0
votes
0answers
25 views

Calculate scores for users with varying activity

I have a web application which allows users to take questions. Ultimately I store the following data: 1) Questions attempted by each user 2) Questions correct by each user 3) Avg. Time per user for ...
1
vote
0answers
28 views

how to compare partial ratings of a set of data by different groups of participants?

So lets say I have N items and I want to rate their relevance (say on a 5-point likert scale: 1 being least and 5 being most relevant) to the "perception of beauty" and I have M number of ...
1
vote
1answer
27 views

Can I split my data into several groups to be rated by various participants?

I am conducting a study on a pool of data which consisted of 200 items. I am planning to use rating scales to see how these items relate to a concept (e.g. "how are the next items related to the ...
1
vote
0answers
133 views

Item ranking algorithm [closed]

I am developing a website with a lot of different items that can be reviewed and rated, although I need a solid ranking algorithm which I can sort after. The ratings will be performed by the user ...
1
vote
1answer
242 views

How to interpret the capped binomial deviance as rating model fit in two player games?

In two player games using rating algorithms like Glicko, the capped binomial deviance is being regarded as a measure of the model data fit. Does anyone know how to interpret this measure? What ...
1
vote
2answers
207 views

How can I estimate a “confidence” in a score / rating

I'm sure there's a more technical term for what I'm looking for, but I don't know what it is (it's not confidence intervals, I think) I'm trying to make a system where people get ratings (from 1 to 5 ...
5
votes
1answer
73 views

adjust rating based on number of experiences

Suppose I am using a restaurant rating as an explanatory variable in a regression. The rating is defined as $R=\frac{G}{G+B+N+S}$, where $G$ is good, $B$ is bad, $N$ is neutral and $S$ is silent. I ...
0
votes
1answer
185 views

Simplest test for consistency of a rating scale

I want to explain to a non-novice doctor the best way to validate consistency of a measure across a likert-scale. The variable describes an average dose across 5 levels of patients (lower to higher). ...
1
vote
0answers
35 views

Detecting and preventing collusion in ratings

I am not sure whether or not this problem can be solved through statistical methods, but here is the problem: I have 4 professional reference providers for a job candidate rating the candidate on a ...
5
votes
2answers
226 views

Fixed-sum question

I have a set of 10 items each with a weight between 0-1. A person can rate each of these 10 items with a score between 0-100, but is given, say, only 500 points to allocate among these 10 items The ...
2
votes
0answers
61 views

Relative scoring system without much effect from outliers

I am trying to assign scores to phone's specifications to calculate a final rating score for each phone. However, when I was scoring based on megapixels. I found that one of the phones 'A' has a 41MP ...
3
votes
1answer
59 views

Is there a data decay method that decays data based on how much new data it receives?

So here's the situation: There are n balls. Each ball has been voted on a scale of 1-N by an arbitrary number of people. Every day (or every week, or more), some/most/all of the previous voters and ...
0
votes
1answer
268 views

Is there a statistically good way to split a set of data into (generally) uneven groups?

So here's the scenario: I have about 100 or so "items", each assigned with a number and total guesses... this number is a guess of what people think the item is worth (in terms of money). This number ...
15
votes
4answers
997 views

How to get started with rating and ranking based on pairwise competition data?

I'm interesting in learning about how to rate and rank individuals in a group who only interact/compete in a pairwise fashion (i.e., systems like the ELO rating system for chess). Are there are any ...
0
votes
0answers
54 views

How to develop a smart sort ordering of list data using popularity/rating algorithms

Here is the problem. We have list of Job orders - each job has certain state associated with it, you can think of state as a situation that job is in, for example, a job that is said to be done or ...
1
vote
0answers
47 views

How to calculate ROC(Ag) in this sound perception study?

I am studying individual differences in sound perception where I ask participants to listen to a pair of sounds and judge whether they are the same or different. The answer choice and scoring are as ...
2
votes
0answers
109 views

Do variations of ELO system exist for non-mirror games?

The ELO rating system and later improvements like Glicko are specifically designed, as far as I know, for mirror games. That is, we assume that no player has a systematic advantage over another. In ...
6
votes
3answers
2k views

How to analyse a ranking and rating scale together?

In some market research consumers are asked to rank the features of a product based on priority. For example, Rank the following features for a device based on your priority (1 being the topmost ...
0
votes
1answer
193 views

How to determine the accuracy of a statement?

Scenario: Consider a statement, e.g. "This movie is an action" Then let people vote on that. 1-5 where 1 is "Not action", 3 is "Some action" and 5 is "Pure action". Question How does one determine ...
1
vote
2answers
161 views

Intersubject concordance for ratings vs rankings

In my system I ask $n$ people to rate [1,5] $m$ objects (e.g. films). I'd like to get a measure of the inter-subject concordance, i.e. how much the users agreed overall, not just on a particular ...
1
vote
3answers
1k views

What is the most effective way to combine different rating scales for the same data?

I have a collection of data which is inherently the same (would expect it to follow the same distribution) where some of it is rated with one scale of ordinal ratings and the rest of the data is rated ...
3
votes
2answers
433 views

Combining ratings from multiple raters of different accuracy

I have a data set of J raters who each give ratings to I objects (on say a scale of 1 to 5). My goal would be to construct some kind of overall rating for each i based on all of the raters' scores. ...
8
votes
1answer
189 views

How to fairly determine winners for a regional science fair?

I need help figuring out the correct way to calculate winners at our science fair. I don't want my ignorance of statistics & math to get in the way of a kid's chances of winning. (lots of ...
2
votes
3answers
196 views

Aggregate opinions to a single measure

I have a following situation in which 6 experts are asked to give opinion about a bottle of wine by rating it from 1 - 5 stars. In addition, somehow we have a measure of the informativeness of each ...
1
vote
1answer
71 views

How to determine if application acceptance is “fair”

I'm a part of a volunteer organization that organizes a bunch of events. For each event, members need to submit an application in order to attend the event. A lot of people want to go to these ...
1
vote
0answers
87 views

Influences of scored/weighted ratings

I am experimenting with a scheme for aggregating $N$ scored ratings. The gathered ratings themselves are integers, e.g. a chosen value between $1$ to $5$, and each rating $r_i$ is assigned with a ...
2
votes
2answers
520 views

How do I sort an ordinal list of user-generated ratings data?

The Ubuntu Software Center uses a 1 to 5 star rating system for its App Reviews. However, it's current ratings sorting algorithm looks very fishy. I believe this is a different question from this ...
3
votes
2answers
190 views

Specification of a random slopes model with two grouping factors

Suppose people take a 10-item exam. For each item, $k = 1,2,…,10$, exactly one rater assigns a score to exactly one person, with the constraint that no person sees the same rater twice. There are $i = ...
4
votes
3answers
367 views

Dissimilarity rating experiment: how to analyze the results

Study: I simulated some surfaces materials at audio and haptic level, and I asked subjects to evaluate on a 9-point Likert scale the degree of coherence between the two stimuli. For example there are ...
4
votes
4answers
368 views

Rating system taking account of number of votes

I need an idea for a new rating system.. the problem in the ordinary one (just average votes) is that it does not count how many votes... For example consider these 2 cases: 3 people voted 5/5 500 ...
10
votes
4answers
510 views

How to prove that Elo rating or Page ranking have a meaning for my set?

I have a set of players. They play against each other (pairwise). Pairs of players are chosen randomly. In any game, one player wins and another one loses. The players play with each other a limited ...
7
votes
2answers
1k views

Weight a rating system to favor items rated highly by more people over items rated highly by fewer people?

Thanks in advance for bearing with me, I am not a statistician of any kind and don't know how to describe what I'm imagining, so Google isn't helping me here... I'm including a rating system in a web ...
2
votes
3answers
774 views

Normalizing rating in a group of people [finding effectiveness]

I'm a new guy here. Hopefully, I'm asking this question to right forum. Problem: We have data of a group of people (P1, P2, P3). They rank their expertise (1-10, where higher number is better) in a ...
10
votes
2answers
250 views

Probability that someone will like image

I've got following problem: - We have set of N people - We have set of K images - Each person rates some number of images. A person might like or not like an image (these are the only two ...
9
votes
3answers
462 views

How do I order or rank a set of experts?

I have a database containing a large number of experts in a field. For each of those experts i have a variety of attributes/data points like: number of years of experience. licenses num of reviews ...
6
votes
3answers
2k views

How would YOU compute IMDB movie rating?

I'm doing this only for learning purposes. I've no intentions of reversing the methods of IMDB. I asked myself I owned IMDB or similar website. How would I compute the movie rating? All I can think ...