Questions tagged [named-entity-recognition]

Named-entity recognition (NER) (also known as entity identification, and entity extraction) is a subtask of information extraction that seeks to locate and classify elements in text into pre-defined categories such as the names of persons, organizations, locations, expressions of times, quantities, monetary values, percentages, etc.

Filter by
Sorted by
Tagged with
1 vote
0 answers
13 views

What is a good approach to annotate invoice numbers for NER?

I have a task for Document object detection. From bank reasons I need to extract several things. One of these things are invoice numbers. The issue with invoice numbers is that one reason can contain ...
  • 131
0 votes
0 answers
18 views

Transition-Based Chunking for NER Task

I am trying to understand how the spacy.TransitionBasedParser.v2 architecture works in SpaCy when running custom training to run a NER Task. After doing a lot of research on the web, I found that this ...
0 votes
0 answers
14 views

Training loss first fall then rise when training a bidirectional LSTM model for a NER task

I'm training a bidirectional-LSTM (Bi-LSTM) + conditional random field (CRF) model for a named entity recognition task. The training set contains 7033 labeled sentences, and the validation set for ...
0 votes
0 answers
14 views

Why are surrounding token embeddings not considered in (simple) NER models?

I just had a look at the hugggingface implemenation of DistilbertForTokenClassification. The token classifier uses the embeddings as input and consists of only one dropout layer and a linear layer. ...
0 votes
0 answers
27 views

Using NER to classify messages based on keywords and rank them by importance

I am new to NLP. I am trying to classify a set of messages based on specific keywords based on importance of a user. For example, you would be given a list of messages and then a person would come and ...
0 votes
0 answers
19 views

Finding a tag / multiple tags across multiple documents

Suppose we have text documents: ...
  • 101
0 votes
1 answer
146 views

BIO vs. BIEWO tagging scheme

In one of my classes, the lecturer explains the tagging scheme BIO (begin, inside, outside) which I knew of and I perceive as the standard. Then he introduced BIEWO which additionally has a tag for ...
2 votes
0 answers
142 views

Is NER suitable for selecting an entire sentence as an entity?

If I have a document with many paragraphs of text, would using a custom NER model be suitable to identify a sentence as a recognized entity? The desired sentences will be similar in semantic structure-...
  • 31
3 votes
1 answer
400 views

Best practice for named entity recognition on large texts

What are the best practices to apply NER to large texts (e.g 20 pages+)? One common advice is to split the text before passing it as input to the model. However this can require a significant manual ...
  • 492
2 votes
2 answers
278 views

Named entity recognition with only one pure entity(no context)?

We know that we can extract entities from a sentence using named entity recognition, but what if the sentence contains only an entity and no other context? For example, we can use CRF for the ...
  • 5,446
0 votes
1 answer
248 views

Named Entity Recognition with Bert on very long Italian documents

As the title suggests, I'm wondering if it's feasible to use Bert to solve the Entity Named Recognition task on long legal documents (> 50.000 chars) in Italian. Now I'm using Spacy, and I'm ...
5 votes
2 answers
2k views

Evaluation metric for named entity recognition

I was watching Manning's lecture on evaluation of NER models, and I'm confused why between 4:07 and 5:13, he states that the error in not labeling one word in a sequence while correctly labeling the ...
0 votes
1 answer
488 views

While evaluating a named entity recognition system, why does partial matching seem to be important?

Through some internet sources, I read that partial matching of classes is also important for finding the precision and recall of a Named entity system. Why is it so?
0 votes
0 answers
62 views

Approach for differentiating "PEOPLE" in Named Entity Recognition

I am using Spacy's pertained model to identify people in the IMDB movie reviews dataset. While the model identifies people in the reviews, I want to find out people who are characters of the movie as ...
0 votes
1 answer
154 views

Does Alphabet Case Matters while Training Stanford Open NLP NER classifier?

I'm working with Named Entity Recognition for non-English. I've some raw text file (all small letter) and trying to make NER classifier. I'm not sure If it'll be better using Small Capital mixed text ...
1 vote
1 answer
128 views

Referring multiple names to the same entity

I am working on the models of different product types and wish to generalize them to the same entity. For example, from the given list ...
  • 113
1 vote
1 answer
1k views

NER at sentence level or document level?

Should NER models (LSTM or CRF) take input training data at sentence level or paragraph level? Let's say we have this input text, and we would like to do Named Entity Extraction: GOP Sen. Rand ...
  • 63
1 vote
0 answers
115 views

Identifying locations in a difficult OCR-read English text with python

My goal is to identify city and (US) state of both inventor and assignee of US patents from the 1910's and 1920's. These patents are provided by google and look like so, like so or like so. The ...
  • 405
1 vote
0 answers
28 views

Sequence Tagging with Incomplete Labels

I have a few thousands of sentences with their B/I/O NER tags. I also have access to millions of sentences for which only some words are tagged. That is, for the other words, I don't know whether they ...
  • 199
1 vote
0 answers
14 views

Ranking Text Posts based on NER Tags

I have a list of NER tags ranked by the number of times they appear in a database of articles, with the highest count listed first. I am trying to figure out how to score the articles based on: How ...
0 votes
1 answer
102 views

Calculating the accuracy of a machine learning system in runtime

I have a question in mind about machine learning systems. Say, I have a named entity extraction systems which give me an accuracy of 90%(precision 90, and recall 90) during training and testing. How ...
1 vote
0 answers
117 views

latent dirichlet allocation for name disambiguation

I'm trying to implement latent dirichlet allocation on a name disambiguation project. My data set includes a corpus of documents. Each document looks like: Author, co-author, title, institution I ...
0 votes
1 answer
66 views

Convolutional Neural Networks for identification of specific objects instead of object categories?

Is it for example feasible to hand a query image of my dog "Dave" to a CNN, and then use the CNN to find all images of "Dave" in a database of dog images? If so, how would one do this? If not, why ...
2 votes
1 answer
792 views

Named entity recognition (NER) features

I'm new to Named Entity Recognition and I'm having some trouble understanding what/how features are used for this task. Some papers I've read so far mention features used, but don't really explain ...
  • 153
2 votes
1 answer
338 views

Is there any advantage of using MEMM instead of CRF for named-entity recognition?

I wonder whether there is any advantage of using maximum-entropy Markov model (MEMM), a.k.a. conditional Markov model (CMM) instead of using conditional random fields (CRF) for named-entity ...
3 votes
1 answer
907 views

Reinforcement Learning & Text Mining

I was wondering if one could use Reinforcement Learning (as it is going to be more and more trendy with the DeepMind & AlphaGo's stuff) to parse and extract information from text. For example, ...
  • 3,988
5 votes
1 answer
966 views

Sequence length when training a conditional random field (CRF)

I am training a conditional random field (CRF) to perform named entity recognition. I have 1000 documents, each containing from 100 to 500 sentences. During the training phase, is it better to train ...