Questions tagged [information-retrieval]
The information-retrieval tag has no usage guidance.
44
questions with no upvoted or accepted answers
6
votes
0
answers
3k
views
Typical range of values for TFIDF
I am working on a text corpus. Each line contains between 10 and 50 words. There are around 25 000 words in the whole text and 1 000 000 lines. I turned this corpus into its tf-idf representation.
I ...
5
votes
0
answers
1k
views
Learning to Rank: query-dependent vs. query-independent features
I've been doing some reading about learning to rank - specifically lambdaMART - and one thing I am confused about is the role of features. When training a model, should one only use query-dependent ...
4
votes
0
answers
1k
views
Why are ERR (Expected Reciprocal Ranking) scores not normalized?
It seems to me that normalized ERR (Expected Reciprocal Ranking) scores (ERR scores of your ranking algorithm divided by ERR score calculated for the ground truth ranking) are more useful than the ...
3
votes
0
answers
81
views
How can I deal with the mismatch between the vocabularies of questions and answers in a closed domain QA system?
I am building a question answering system that given a legal document attempts to answer questions related to the document. For example a tenancy agreement is given to the system and the user asks ...
3
votes
0
answers
112
views
Heuristics streaming data matching
I have an index composed by thousands of documents. Slightly modified copies of those documents are sent to my application in small chunks, and I need to check, from those chunks, which document has ...
3
votes
0
answers
121
views
Evaluation of a semi-supervised ranking model
I learned ranking model with graph-based semi-supervised approach, while labeled (just positive) and unlabeled (positive and negative) data is both used in training. With the model, all of the data ...
3
votes
0
answers
445
views
Similarity calculations for arrays
First of all, my apologies if I mess up the terminology. I've been out of math for several years, so I'm certain I'm going to use terms incorrectly. Also, though I concentrated mathematics in college, ...
2
votes
0
answers
13
views
Looking to extract patterns from sequences of codes
I have the following problem: I have a registration of people who enter a building, I have the name, entry date and end date. I also have the times at which events occur inside the building.
I want to ...
2
votes
0
answers
20
views
Extracting information from bills, tax statements, etc: What ML model to use?
I have a bunch of documents such as bank statements, utilities bills, personal expenditure invoices, etc. The document types range is very broad. Some of these files are saved as pictures, others as ...
2
votes
0
answers
545
views
Metrics for Multilabel Classification
From what I've read, F1-score is a commonly used metric to assess the performance of a multilabel classification problem.
However, I recently came across mAP@K and mAR@K as metrics used for ...
2
votes
0
answers
40
views
mislabeled data Event Detection, neural language processing
I'm doing some experiment with sequence labeling problems, more specifically is Event detection problem. But I'm encountering with the mislabeled data issue.
In my dataset, there are many tokens that ...
2
votes
0
answers
90
views
What is an appropriate Evaluation Metric and corresponding Loss function which best optimize the metric for a classification based FAQ Chatbot?
I am developing a FAQ chatbot to display/return only one correct answer in a chat window for a given question from the user.
I know MRR & MAP make sense as an evaluation metric for information ...
2
votes
0
answers
99
views
F-measure and hypothesis testing
I would compare two classifiers (A and B), where B is obtained from A.
I would exploit the F-measure computed on two samples (of two independent populations, p1 and p2, respectively):
A -> F-...
2
votes
0
answers
758
views
Is there an algorithm for determining scoring function (Utility function) in ranking instances?
So while going through the topic of Preference learning, I came to know about "instance ranking". Since the problem which I'm working on requires me to rank the instances (data point), is there any ...
2
votes
0
answers
146
views
Search in a graph of medical records
I have a hierarchical order of a disease list. Just for example:
Respiratory system disease
1.1. Asphyxia neonatorum
1.2. Croup
1.3. Lower respiratory tract disease
1.4. Bronchial disease
1....
2
votes
0
answers
121
views
Software library for Hidden Markov Modeling of a large text database
Given
we have a large database of texts (e.g. product descriptions)
and we want to extract multiple types of information (e.g. brand, release date, features, price, etc.)
what's a good library to ...
1
vote
0
answers
37
views
Calculating co occurrence probabilities of search queries
Hi guys I want to calculate the pointwise mutual information for related search queries on an e-commerce website.
In order to calculate that I need to fist calculate the co occurrence matrix for the ...
1
vote
0
answers
34
views
AUC-like measure for multiple simultaneous classification tasks?
I know that given an ordered set of binary labels, and equally-sized ordered set of scalar predictions, we can quantify how cleanly the predictions separate the labels into clean buckets of 0's and 1'...
1
vote
0
answers
423
views
Youden's J statistic can be negative but it is said to be between 0 and 1
J = (true_positives x true_negatives - false_positives x false_negatives) / (positives x negatives)
where positives and negatives are the number of real positive and real negative samples.
...
1
vote
0
answers
47
views
ML Approach for Optimizing Field Boosts for Search (Information Retrieval)
I have been experimenting with different methods for tuning our search engine's field boosts. In Solr or Elastic Search, you specify the importance of matches in each field when configuring the search ...
1
vote
0
answers
235
views
Learning to rank and traditional information retrieval evaluation
I have some questions about best practices in information retrieval (unsupervised) vs learning to rank evaluation.
How necessary in a train-validation-test or cross-validation scenario? is it ...
1
vote
0
answers
417
views
NDCG for recommender algorithm
I need to apply NDCG over the results of a recommender algorithm, but was not able to find any proper example that suits my use case in order to find out if my implementation is correct.
Here is my ...
1
vote
0
answers
102
views
Representing Text for Section Labeling While Avoiding Bias
Problem Background: I have free form text data with more or less arbitrary formatting/structure, but semantically it can be broken down like this:
...
1
vote
0
answers
302
views
Why F-1 is a better score than the harmonic mean between true negative rate and recall?
Beside of the easy interpretation, F1 measure is very sensible to the relative frequency between positives and negatives. ROC AUC metric don't, but it is difficult to optimize. Are there any special ...
1
vote
0
answers
456
views
Union of good feature sets degrades accuracy
I am doing binary text classification and I have some feature sets (unigrams, bigrams, dependencies, etc.) and each one of these performs very good individually. For example unigrams alone achieve 89% ...
1
vote
0
answers
340
views
Contextual Matching algorithm implementation in Python
I have been trying to implement an algorithm using Python in order to perform contextual matching in a set of documents. My ultimate goal would be to be able to perform queries using positive keywords ...
1
vote
0
answers
28
views
Average Number of Documents Containing a Term
I am working on a research project in which I am using inverted index for terms and documents in one of the techniques. I am doing algorithmic analysis for all the methods so that I can compare and ...
1
vote
0
answers
52
views
How to decide what is the relevant group in a precision and recall computation?
One of the most famous measurements for an information retrieval system is to compute its precision and recall. For both cases, we need to compute the number of total relevant documents and compare it ...
1
vote
0
answers
240
views
Is this ordering of examples redundant in Rank SVM?
I was studying RankSVM[1] and I was observing the ranking pairs listed in the example here. Formulation can be seen on page 4 of this paper.
One of the ordering is:...
1
vote
0
answers
30
views
Information retrieval performance measures for unknown test collection?
I am evaluating a web search relevance feedback algorithm. The algorithm uses Bing API as source for its result sets. To evaluate the algorithm I will be conducting a user study.
In the end I will ...
1
vote
0
answers
165
views
MAE/MSE with or without square root
I read some papers about recommender systems and information retrieval, where Mean Absolut Error and Mean Squared Error are mentioned. But I've found some differences between the formal definition of ...
0
votes
1
answer
764
views
(n)DCG without true relevance scores
I have a search engine that returns results with a normalized score on the scale 0.0 to 1.0. Higher score means higher relevancy to the input query.
The ranked output scores look like this, e.g. [1.0, ...
0
votes
1
answer
28
views
Extracting information from form document through supervised learning
I was searching for a while around the web and I couldn't find any solution that would give some ideas on how to solve my problem. I have a few hundreds of document with some permission forms filled ...
0
votes
0
answers
44
views
What is the standard metric used in recommendation systems to evaluate the rankings?
I was searching for a metric to do this for a while and still could not find.
More specifically, my problem is as follows.
I have a ranked golden corpus. For example, consider that it looks as ...
0
votes
0
answers
485
views
Weighting for precision and recall
I want to integrate the notion of weighting into an evaluation. I am wondering if it is appropriate/correct to calculate precision and recall scores by adding a weighting on true positives, false ...
0
votes
0
answers
31
views
What are most recent research work on the problem of key phrases extraction from a text corpus?
I am interested in the problem of extracting key phrases from a text corpus. This is different from the keyword extraction problem, which is only for a particular document. This problem helps us, for ...
0
votes
0
answers
26
views
Estimating a surprise of a word in context
What will be the best way to estimate the entropy/surprise of a word in a specific context?
Let's say to compare the surprise of:
context:
"I watched the movie in my"
word:
Computer
I ...
0
votes
0
answers
112
views
What is a rotation matrix and how to implement it?
In Revisiting the VLAD Image Representation the authors introduce Local Coordinate System, i.e. they:
we learn off-line (for each visual word) a rotation matrix Qi from training descriptors mapped ...
0
votes
1
answer
49
views
Where do initial document values come from in K-means document clustering?
So the K-means algorithm seems simple enough as I understand it: given some documents, turn those documents into points, initialize some number of k (centroids), assign document-points to nearest ...
0
votes
0
answers
300
views
topic modeling for image retrieval
I'm interested in learning a topic model from a bag of visual words for image retrieval. I can compute V cluster centers (visual words) of SIFT descriptors at keypoints for each training image and fit,...
0
votes
0
answers
78
views
An IR evalualuation metric that only measures the rank of results?
I am working on a little text clustering problem, and trying to figure out how to evaluate the results. I came up with the following idea that I though fits pretty well with the specifics of the ...
0
votes
0
answers
299
views
Is precision in recommender system related to mean average error (MAE)?
A recommender system is being evaluated while increasing the neighborhood size. The highest precision was achieved between 10-15 neighbors(users) while the lowest MAE was in the range from 30-40 users....
0
votes
0
answers
27
views
Evaluation of IR approach without test collections
I am trying to evaluate an information retrieval approach. More specifically, it's a query expansion algorithm, based on topic distribution in retrieved documents.
I want to evaluate my approach ...
0
votes
0
answers
106
views
Algorithms for keyphrase clustering
Are there any standard algorithms for keyphrase clustering. There are several algorithms for keyphrase extraction from a corpus. For e.g. this publication reviews some of the popular keyphrase ...