Questions tagged [information-retrieval]
The information-retrieval tag has no usage guidance.
115 questions
0
votes
0
answers
9
views
Does it make sense using nDCG with implicit feedback?
I am comparing multiple offline recommender systems models on an implicit feedback dataset and reporting various metrics.
These models follow the same order with multiple metrics. The best model is ...
- 101
2
votes
0
answers
15
views
In the context of document retrieval, what serves as the ground truth?
In the context of the benchmark datasets that I'm using for document retrieval, the samples are usually comprised of a query and its corresponding positive and negative passages. A positive passage ...
- 4,177
2
votes
0
answers
17
views
Looking to extract patterns from sequences of codes
I have the following problem: I have a registration of people who enter a building, I have the name, entry date and end date. I also have the times at which events occur inside the building.
I want to ...
- 205
1
vote
0
answers
54
views
Calculating co occurrence probabilities of search queries
Hi guys I want to calculate the pointwise mutual information for related search queries on an e-commerce website.
In order to calculate that I need to fist calculate the co occurrence matrix for the ...
2
votes
1
answer
95
views
Why does my bi-encoder converge to the mean square of the [0,1] label distribution? [duplicate]
I'm following the Bi-Encoder architecture (see here) in order to build a dense retrieval (search) system. Formally, my network encodes a query q and an item description d based on fixed ...
- 21
2
votes
0
answers
21
views
Extracting information from bills, tax statements, etc: What ML model to use?
I have a bunch of documents such as bank statements, utilities bills, personal expenditure invoices, etc. The document types range is very broad. Some of these files are saved as pictures, others as ...
- 5,882
0
votes
1
answer
927
views
(n)DCG without true relevance scores
I have a search engine that returns results with a normalized score on the scale 0.0 to 1.0. Higher score means higher relevancy to the input query.
The ranked output scores look like this, e.g. [1.0, ...
- 101
0
votes
1
answer
34
views
Extracting information from form document through supervised learning
I was searching for a while around the web and I couldn't find any solution that would give some ideas on how to solve my problem. I have a few hundreds of document with some permission forms filled ...
- 101
3
votes
1
answer
3k
views
False positive rate at K recall
I just stumbled upon a new metric I've never heard about called False Positive rate at K recall (FPR-K).
Searching the internet I just managed to find more papers using the metric but none of them ...
- 241
2
votes
0
answers
561
views
Metrics for Multilabel Classification
From what I've read, F1-score is a commonly used metric to assess the performance of a multilabel classification problem.
However, I recently came across mAP@K and mAR@K as metrics used for ...
- 133
1
vote
1
answer
160
views
Real vs True Positives
Wikipedia defines TPR (True Positive Rate) as $\frac{\text{TP}}{P}$
where:
$\text{TP}$ = # of true positives
$\text{P}$ = # of real positives
This confuses me.
I thought:
$\text{TP}$ is supposed ...
- 4,598
2
votes
0
answers
40
views
mislabeled data Event Detection, neural language processing
I'm doing some experiment with sequence labeling problems, more specifically is Event detection problem. But I'm encountering with the mislabeled data issue.
In my dataset, there are many tokens that ...
- 21
0
votes
0
answers
54
views
What is the standard metric used in recommendation systems to evaluate the rankings?
I was searching for a metric to do this for a while and still could not find.
More specifically, my problem is as follows.
I have a ranked golden corpus. For example, consider that it looks as ...
- 602
1
vote
0
answers
34
views
AUC-like measure for multiple simultaneous classification tasks?
I know that given an ordered set of binary labels, and equally-sized ordered set of scalar predictions, we can quantify how cleanly the predictions separate the labels into clean buckets of 0's and 1'...
- 2,049
0
votes
0
answers
518
views
Weighting for precision and recall
I want to integrate the notion of weighting into an evaluation. I am wondering if it is appropriate/correct to calculate precision and recall scores by adding a weighting on true positives, false ...
- 101
2
votes
1
answer
39
views
Models for ranking possible classifications by confidence
What are good means of finding the various highest confidences or likelihoods for a (say) hundred possible outcomes of a classification problem?
Inputs belong to only one class, unlike document ...
- 211
1
vote
0
answers
485
views
Youden's J statistic can be negative but it is said to be between 0 and 1
J = (true_positives x true_negatives - false_positives x false_negatives) / (positives x negatives)
where positives and negatives are the number of real positive and real negative samples.
...
- 111
0
votes
0
answers
33
views
What are most recent research work on the problem of key phrases extraction from a text corpus?
I am interested in the problem of extracting key phrases from a text corpus. This is different from the keyword extraction problem, which is only for a particular document. This problem helps us, for ...
- 1
0
votes
2
answers
169
views
How is Cross validation used with non-machine learning problems?
I am fairly new in the field of Information retrieval. I have basic knowledge about machine learning. I understand the purpose of CV in the context of Machine learning. However, I've become a bit ...
- 1
2
votes
0
answers
91
views
What is an appropriate Evaluation Metric and corresponding Loss function which best optimize the metric for a classification based FAQ Chatbot?
I am developing a FAQ chatbot to display/return only one correct answer in a chat window for a given question from the user.
I know MRR & MAP make sense as an evaluation metric for information ...
- 5,683
0
votes
0
answers
26
views
Estimating a surprise of a word in context
What will be the best way to estimate the entropy/surprise of a word in a specific context?
Let's say to compare the surprise of:
context:
"I watched the movie in my"
word:
Computer
I ...
- 57
2
votes
2
answers
120
views
What are commonly used methods to represent a document by a vector?
Methods that I know of
Bag of words + weighting: tf-idf, bm25
Topic models: LSA, LDA
Word/sentence/document embedding
Are there other commonly used methods to represent a document by a vector?
- 16.8k
1
vote
0
answers
47
views
ML Approach for Optimizing Field Boosts for Search (Information Retrieval)
I have been experimenting with different methods for tuning our search engine's field boosts. In Solr or Elastic Search, you specify the importance of matches in each field when configuring the search ...
- 288
4
votes
1
answer
3k
views
Remove bias in ranking evaluation
In ranking, a commonly used method to evaluate the performance of a ranking algorithm is calculating ranking metrics such as MAP, NDCG. In use cases where there is no ground truth signal, some proxy ...
- 151
3
votes
2
answers
833
views
inverting the binomial distribution: probability distribution for number of trials necessary to have a given number of successes
The binomial distribution gives me a distribution for the number of successes in several Bernoulli trials, k, given parameters N the total number of trials and q, the success probability for one ...
- 615
1
vote
0
answers
235
views
Learning to rank and traditional information retrieval evaluation
I have some questions about best practices in information retrieval (unsupervised) vs learning to rank evaluation.
How necessary in a train-validation-test or cross-validation scenario? is it ...
- 71
1
vote
1
answer
86
views
What contests/datasets for expertise retrieval ?
Expertise retrieval is a difficult task to define. This article describes the task of expert finding as "finding the right person with the appropriate skills and knowledge". Applications exist in ...
- 236
0
votes
1
answer
478
views
Extracting Part of Speech (Source and Destinations) using text mining/NLP?
I need to extract the source and destination terms from the text
documents using text mining/NLP/Information Retrieval ?
ex :
i am travelling from New York to London.
i am heading towards ...
- 111
1
vote
0
answers
419
views
NDCG for recommender algorithm
I need to apply NDCG over the results of a recommender algorithm, but was not able to find any proper example that suits my use case in order to find out if my implementation is correct.
Here is my ...
- 11
0
votes
3
answers
4k
views
What algorithm should I use to predict a continuous dependent variable from multiple continuous & categorical independent variables?
I'm software engineer of an E-commerce company, facing a problem like this:
An e-commerce shop sells their products daily and wants to know what conditions that might improve their sales. I'm ...
4
votes
1
answer
4k
views
Both validation loss and accuracy goes up in neural network
I'm training a 2-layer CNN model on audio samples, represented as CQT. There are ≈160k samples, many that are very similar since they originate from the same instrument and/or audio file. 10% have ...
- 41
3
votes
0
answers
86
views
How can I deal with the mismatch between the vocabularies of questions and answers in a closed domain QA system?
I am building a question answering system that given a legal document attempts to answer questions related to the document. For example a tenancy agreement is given to the system and the user asks ...
- 31
2
votes
0
answers
173
views
How to adjust your dataset so it fits a power-law? [closed]
I have created a dataset of pictures taken at the museum of different paintings. The dataset is divided into 113 different categories (paintings) and contains around 4.8k images.
Just to be clear: ...
- 399
1
vote
0
answers
103
views
Representing Text for Section Labeling While Avoiding Bias
Problem Background: I have free form text data with more or less arbitrary formatting/structure, but semantically it can be broken down like this:
...
- 21
3
votes
1
answer
2k
views
How to find "similar documents" after a Latent Dirichlet Allocation model is built
Let's say I run an LDA model with 3 topics on 5 documents.
After the model is learned (with Gibbs sampling presumably), I have topic distribution for each document, shown as the following:
My ...
- 799
2
votes
2
answers
13k
views
Drawbacks with Cosine Similarity
I am assessing the similarity between documents represented as vectors of tf-idf values. I know that the cosine similarity is a well-defined and commonly used measure in information retrieval.
...
- 121
1
vote
0
answers
308
views
Why F-1 is a better score than the harmonic mean between true negative rate and recall?
Beside of the easy interpretation, F1 measure is very sensible to the relative frequency between positives and negatives. ROC AUC metric don't, but it is difficult to optimize. Are there any special ...
- 2,098
0
votes
0
answers
113
views
What is a rotation matrix and how to implement it?
In Revisiting the VLAD Image Representation the authors introduce Local Coordinate System, i.e. they:
we learn off-line (for each visual word) a rotation matrix Qi from training descriptors mapped ...
- 399
1
vote
0
answers
459
views
Union of good feature sets degrades accuracy
I am doing binary text classification and I have some feature sets (unigrams, bigrams, dependencies, etc.) and each one of these performs very good individually. For example unigrams alone achieve 89% ...
1
vote
0
answers
348
views
Contextual Matching algorithm implementation in Python
I have been trying to implement an algorithm using Python in order to perform contextual matching in a set of documents. My ultimate goal would be to be able to perform queries using positive keywords ...
- 111
1
vote
0
answers
28
views
Average Number of Documents Containing a Term
I am working on a research project in which I am using inverted index for terms and documents in one of the techniques. I am doing algorithmic analysis for all the methods so that I can compare and ...
- 113
3
votes
2
answers
314
views
Is there a ranking metric based on percentages that favors larger magnitudes?
I have two groups, "in" and "out," and item categories that can be split up among the groups. For example, I can have item category A that is 99% "in" and 1% "out," and item B that is 98% "in" and 2% "...
- 314
2
votes
2
answers
1k
views
Can we use Bag of Visual Words to compute similarity between images directly?
I'm implementing a Content Based Image Retrieval application (CBIR).
I've read about the Bag of Features model and it's considered an intermediate-step algorithm in some application. For example, ...
- 399
2
votes
0
answers
101
views
F-measure and hypothesis testing
I would compare two classifiers (A and B), where B is obtained from A.
I would exploit the F-measure computed on two samples (of two independent populations, p1 and p2, respectively):
A -> F-...
- 156
6
votes
1
answer
5k
views
Difference between Log Entropy Model and TF-IDF Model?
I would like to understand what are the differences/advantages in using TF-IDF or the Log Entropy model for represeting documents and queries in an information retrieval system using diferent weights.
...
- 63
3
votes
1
answer
3k
views
Where did sublinear tf-idf originate?
I have often come across this weighting scheme for tf-idf (term frequency - inverse document frequency) in text mining. I am wondering where it came from (for citations). I've searched very rigorously,...
- 131
0
votes
1
answer
50
views
Where do initial document values come from in K-means document clustering?
So the K-means algorithm seems simple enough as I understand it: given some documents, turn those documents into points, initialize some number of k (centroids), assign document-points to nearest ...
- 101
1
vote
0
answers
52
views
How to decide what is the relevant group in a precision and recall computation?
One of the most famous measurements for an information retrieval system is to compute its precision and recall. For both cases, we need to compute the number of total relevant documents and compare it ...
- 11
0
votes
0
answers
302
views
topic modeling for image retrieval
I'm interested in learning a topic model from a bag of visual words for image retrieval. I can compute V cluster centers (visual words) of SIFT descriptors at keypoints for each training image and fit,...
- 1,168
7
votes
1
answer
4k
views
The Effect of Stopword Filtering prior to Word Embedding Training
Recently I have played with the pretrained GLOVE word embedding model for Twitter
http://nlp.stanford.edu/projects/glove/
I notice that common stopwords are existing in the model. That is, there is ...
- 616