Questions tagged [ranking]

Ranking is the task (for respondents) or the result of ordering the given stimuli or performers from "highest" to "lowest" (or *vice versa*) in some respect. It is usually contrasted with rating of stimuli. (For ranking as a way of data transformation - use tag [ranks]).

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

Compare alternate force choice data to ranking data

I am completing an experiment in which website preferrence were being tested. first the participants were shown 4 images and asked to rank 1-4 for a set of 20 images. secondly, they were displayed 2 ...
0
votes
1answer
27 views

Top-N ranking loss is difficult to optimize directly. Why?

Variational Autoencoders for Collaborative Filtering paper tells: "Recommender systems are often evaluated using ranking-based measures, such as mean average precision and normalized discounted ...
0
votes
0answers
8 views

Dominance matrix for races [closed]

Is it possible to determine a ranking of participants in a series of races through a dominance matrix? For example, each race has 10 participants, there are many races, and not every participant will ...
0
votes
0answers
20 views

How to optimise a linear function where training data only has pairwise relative ranking of data points

I have a data set containing a number of features and want to build a linear prediction model from this. However, I do not have a target score in the training set, instead I only have a number of ...
0
votes
0answers
6 views

sklearn implementation of nDCG seems not suitable for recsys that generate a ranked list of items. Any library recommended?

Most recommenders do one of two tasks. They either ... attempt to predict a rating of an item by a user, or generate a ranked list of recommended items per user. What I want I'm currently interested ...
0
votes
0answers
14 views

Ranking samples of a stochastic function

Suppose we have samples $f_1, f_2, ..., f_n$ of some stochastic function $f(\vec{x}): \mathbb{R}^n \rightarrow \mathbb{R}^1$ for different arguments $\vec{x_1}, \vec{x_2}, ..., \vec{x_n}$. We may ...
0
votes
0answers
18 views

How to scale one dataset in relation to another?

I have two sets of numbers, that represent grades from two different groups of students, A and B, these grades go from 0 up to 1000. These sets have very different distributions, while set A has a lot ...
0
votes
0answers
13 views

Ranking metrics: weighted and different

I have 2 ranking lists and I want to see how similar are they between them. setA = {10,9,8 ,7 ,6,5,4,3,2,1} setB = {4,5,12,14,9,8,7,6,2,1} There are the following constraints: 1. High number (or if ...
0
votes
0answers
11 views

How to add ranking loss to cross entropy loss

I have a multiclass classification problem and I use cross entropy loss to train my DNN. With cross entropy loss, we only take into consideration the top prediction. Is there anyway I can penalize the ...
0
votes
0answers
15 views

How to determine if a list has been ranked on some criterion?

I have a list of a few hundred individuals and two methods which claim to rank the list in descending order of probability of mortality. I also have data to show whether death occurred within a set ...
0
votes
1answer
18 views

How to handle the dummy variables with overlapping categories?

Background of The Question Let's say, I have four categories (A, B, C, D). Considering one (D) as a reference variable, there will be three categories on which I have to work. But the problem is one ...
0
votes
0answers
22 views

Offline Precision@k and Recall@k for recommender system

How can I evaluate offline Precision@k and Recall@k metrics for recommender system if I only have items-users matrix? I think I can't just compare recommendations and user data because it will be ...
0
votes
0answers
4 views

How to compare two different ordinal scoring system for same population. System 'A' has four values, 1-5 while System 'B' has 7 values, 1-9

I have a data of 1000+ observations. I need to compare the output of two different models which gives ordinal scores. Model A gives the values from 1(low risk) to 5(high risk), while the model B gives ...
0
votes
0answers
36 views

Custom Loss Function in Decision Tree for Ranking

I have built a Decision Tree and Adaboost model from scratch in Python and am now trying to customize the loss function being used. I am hoping to use a ranking loss function but am having troubles ...
0
votes
0answers
46 views

Find population characteristics from survey proportions

I want to launch surveys on very specific binary questions with low "Yes" answer rate, for example Do you have a Ferrari ?, Did you buy a Chanel bag ? I'm able to do that on at least 20 ...
0
votes
0answers
15 views

Multiobjective ranking

I am considering a machine learning ranking problem where I have to rank a set of items based on multiple criteria. Item i is represented by a feature vector $x_i \in \mathbb{R}^d$. Item i also has K ...
1
vote
0answers
20 views

How to analyze ranked data (preference)

I'm conducting a user study where I ask respondents to rank different versions of same sentences by their preference. What would be an adequate way to analyze this type of ranked data? Ultimately, I'm ...
0
votes
0answers
22 views

How to compare a ML ranking method to ground truth, emphasizing top ranks? Weighted Pearson correlation between ranks?

I want to compare two methods of ordering (ranking) a set: one is the gold standard / "ground truth". The other is a machine learning model (or more precisely, the order of log probabilities ...
0
votes
0answers
8 views

How to estimate click through rate lift for a ranking model?

Suppose I want to build a ranking model for product search in an eCommerce site. I can use a pointwise or pairwise approach to build a ranking model based on the click data collected under the ...
2
votes
1answer
42 views

How can I compare the variance or correlation between ranked *single answer* polls?

I am an undergrad psych major examining the differences in personal preferences between the extremist website 4chan and other websites, which will represent the pop norm. I would like to test whether ...
2
votes
0answers
30 views

What is a dragon king?

I am studying a system of cities where the largest city appears to be in many aspects an outlier. The distribution of city size - in any country - are often claimed to follow Zipf's law. According to ...
1
vote
0answers
24 views

Calculating the probabilities, for multiple candidates, of ranking first, and where those probabilities sum to 1, using ordinal logistic regression

First time poster on Cross Validated. I'm trying to work out how to predict the winner of a race, of, for example, 5 contestants, where, for each of those contestants, I have 3 attributes, as X's, Age,...
0
votes
0answers
17 views

Creating a scoring system for most favorable outcome; similar to Canada's immigration system

I am trying to create a ranking system similar to Canada's immigration system (which is very well defined) in my field of study. Here is the link to the webpage listing all the rules of this ...
1
vote
0answers
38 views

Bayesian meta-analysis of multiple ranked lists?

Let's say I go around and ask a bunch of my friends to rank 30 movies. Each one returns me a list. Now the obvious treatment is to average the rankings, but I'm wondering if anyone has seen a more ...
1
vote
0answers
20 views

Relevant statistical test for data of a ranking experiment

I'm currently thinking of a relevant statistical test to use to assess the results of a small study. Imagine i have 6 different brand of a food product and i want to know whether the participants to ...
0
votes
0answers
8 views

Find Maximum from Noisy Observations

Problem Setup I observe data from $N$ sources, each generates data indpendently according to its own distribution $\pi_j,\ j=1,\dots,N$. I collect $n$ observations from each source. Thus, I have $X^{(...
5
votes
1answer
105 views

Hypothesis test: numeric vs. ranked

I believed that the most powerful hypothesis test for judging whether a single sample comes from $N(0,1)$ or from $N(1,1)$ uses the average value as test statistics. Thus, I calculate the sample size ...
1
vote
1answer
35 views

How to compare multiple Kendall's tau results?

Looking for a bit of help on how to "aggregate" Kendall's Tau values from a meta-analysis To offer a conceptual view of the data: let's say I have 40 different "studies" that each ...
0
votes
0answers
14 views

Test for whether a binomial outcome is associated with difference in rank in two lists

I have a number of items that are coded using a single dichotomous measure $x$ that reflects success or failure. I have two different rankings of all items ($R_1$ and $R_2$) and would like to conduct ...
0
votes
0answers
26 views

Building a Ranking Model for Junior Golf

So, I'm trying to figure out the best way to build a model for Junior Golf Rankings. The best system out there currently uses a Sagarin type methodology that compares head-to-head records of ...
3
votes
0answers
35 views

How to control the error rate when selecting the mode of a sample as the mode of the population?

I just looked out of my window and saw 4 white, 2 black, 1 green, 1 silver, 1 light blue, 1 lemon and 1 orange-white-striped car. Based on this, how confident should I be that white is the most common ...
0
votes
0answers
15 views

Regression results ranking

I have a XGBoost regression model and I'm using metrics like MAPE,MSE etc. to tune the model and evaluate results. Just for evaluating results, I also want to check if the model ranks my target ...
0
votes
0answers
11 views

Are the Condorcet method and Borda method non-parametric?

I'm having trouble understanding if the Condorcet method and Borda count make the normality assumptions necessary for parametric methods. It seems that by forcing voting, they are okay with ordinal ...
0
votes
0answers
5 views

Combining time and percentage errors into a single figure of merit

I have a set of 1000 randomly generated "model" objects, each of which can be solved using one or more "solver" objects (there are 6 different solvers in total). For each model, I ...
0
votes
0answers
6 views

Comparing two sets of data using two-way comparisons

I have two sets of samples, called $A$ and $B$. I want to find out whether on average samples in $A$ are "better" than in $B$ or vice versa. "Better" is a qualitative vague measure ...
0
votes
2answers
40 views

Scoring a ranking question

I’m tasked with writing exam questions and I want to include a ranking question. The idea is to have students rank 5 items in the correct order. When scoring the question, I want to do partial grading ...
0
votes
0answers
32 views

Discrete choice model with different choice set for each sample

Data I have observational data of $N$ rows (samples). In row $n$ individual $k_n \in \{1, \ldots, K\}$ from a population of $K \ll N$ individuals chooses one option $y^{k_n}_n$ from $J_n$ options, ...
2
votes
1answer
47 views

How can I rank preferences without ranking records in a ML model?

In a binary classification problem about the purchase of a product I use AUC to evaluate the performance of the model. Due to some restrictions I can't assign to each record of my data set any metric ...
0
votes
0answers
19 views

Comparing ranking methods across multiple

I have a bunch of ranking algorithms that rank a collection of $k$ items. I have $n$ of these collections that I want to evaluate each algorithm on. If I didn't have $n$ of these collections I would ...
0
votes
1answer
35 views

What percentile is the following mark?

$24$ students took a test and they each scored a mark from $0$ to $100$. Gillian scored $71$. $18$ students scored lower than her and $5$ students scored higher than her. Gillian is the $19$th mark. ...
1
vote
0answers
7 views

How the pattern ranking performed in this SDM paper?

I am following this SDM paper "Diversified Trajectory Pattern Ranking in Geo-Tagged Social Media" that I found very intersting and inspiring. However due to my limited knowledge in mathematical ...
0
votes
0answers
30 views

How to Build a Pairwise Ranking Model Using Tree Based Algorithms?

Building a pairwise ranking model using linear algorithms is easy to understand. We just need to do pairwise transformation on the original data and create 0/1 labels. Then feed these pairwise ...
1
vote
0answers
34 views

Learning to Rank with binary target variable

Is there a Machine Learning Ranking algorithm that can rank documents for a query using as a training dataset to the algorithm a dataset having: some features of the queries, some features of the ...
0
votes
0answers
12 views

interpret correlation

Excuse the fairly simple question. I am comparing two measures, L1 (city-block) distance, and Kendall Distance over a hypothetical dataset of ranked lists. To understand their relationship, I ...
3
votes
1answer
65 views

Rank-ordered data - dealing with increased randomness among lower ranks

What is the best way to analyze rank-ordered data when there are signs that respondents were less diligent/able to assign lower ranks? Is it sufficient to introduce a dummy for lower/earlier ranks ...
1
vote
0answers
17 views

Positional features and feedback loops in ranking

I read the following in the Google "rules for machine learning": Rule #36: Avoid feedback loops with positional features. The position of content dramatically affects how likely the user ...
0
votes
0answers
6 views

Ranking: Only allowing features to have positive weights

I read the following in the Google "rules for machine learning": Rule #35: Beware of the inherent skew in ranking problems. Only allow features to have positive weights. Thus, any good ...
0
votes
0answers
13 views

Elo, but with vector rankings

Elo gives us scalar rankings: A has 1500 points, B has 1200 points, and as such A will beat B in ~75% of games. In some sports, performance is closer to a vector, where some players are better at ...
1
vote
0answers
17 views

How should I visually represent these preference rankings?

I'm trying to visually represent the following rankings in a clear and intuitive way, but am unsure of how to best do that. Here's the data: There are 4 outcomes: A1, A2, B1, B2 The letters are the ...
0
votes
0answers
23 views

How do ML ranking algorithms like LambdaMART generalize for unseen queries?

I am currently studying Learning to Rank algorithms like RankNet, LambdaRank, LambdaMART. I want to use them to recommend items for a given query. Thus, I want to learn to rank items for queries. I ...

1
2 3 4 5
12