# Tagged Questions

12 views

### choosing best value for N when using N-Gram approach

the question is quite general, but I am doing a research related to supervised machine learning to classify two set of characters into two categories. in fact, I want to compute some measures of ...
5 views

### Get positions/values from heat costs bill

for a project I need to extract values from customers yearly heat costs bill. The customer takes a photo of the bill and the program should extract the values heating period of the billing, type of ...
39 views

### Using relative frequency for Euclidean and cosine distance (dissimilarity)

How to calculate the Euclidean distance (dissimilarity) between two documents, e.g., D1 and D2 using relative frequency? Here is an example of both cosine and Euclidean distance between two ...
78 views

### Weighting words based on position in text

I'm currently working on semantic analysis and had a question about text organization and structure. Are there any algorithms, or statistical / machine-learning models that weight the importance of a ...
36 views

### How do I perform an IDF calculation?

How do I perform an IDF calculation? I am uncertain as to whether IDF should be calculated in per-class level or for the entire document set (that contains multiple classes).
31 views

### What are the mathematics I need to learn, before I start research in data mining [duplicate]

I usually use text mining, graph mining, Information retrieval, and natural lanuage processing. Also i will use the fundamental concepts of data mining like classification, association and clustering. ...
107 views

### Create a matrix of tf-idf values from documents

I have a set of documents like: D1 = "The sky is blue." D2 = "The sun is bright." D3 = "The sun in the sky is bright." and a ...
68 views

### Clustering structured data: Assessing the similarity of documents that appear in tree structure

Usually when performing text document clustering, similarities across documents are assessed based on the lexical content of documents. But, in my problem, I wish to consider both the lexical content ...
23 views

### LexRank damping factor

I am looking into using LexRank to do some text summarization. I am looking at the original paper. One thing that puzzles me is whether a damping factor is used or not. The formulae are all using it, ...
52 views

### Automatic labeling of training set

I have once meet the following question, given a training set, is that possible to do the automatic labelling? In addition, if this training set consists of plain text files, is that possible to know ...
121 views

### Dataset and papers for baseline [closed]

I'm doing a project about Topic Detection and Tracking in text. I need to perform a baseline so I can compare existing results with mine. I read some papers where they use datasets that are not so ...
30 views

### “Exotic” text representations methods?

I'm looking to the different methods of representing a text into a machine-readable format. However, until now, I only found "Bag of Words" approachs with a lot of variations (boolean BoW, weighted ...
155 views

### Is there any dataset or api that gives a list of infrequent words? [closed]

I'm actually working on an information retrieval project, and I want to extract words that are of significance from a sentence. This is somewhat opposite to stopwords. In a sentence like: He was a ...
2k views

### Euclidean Distance b/t unit vectors or cosine similarity where vectors are document vectors

I was reading Similarity Measures and suddenly my whole world was falling apart. I have implemented a search engine using clustering techniques. For clustering, I used k means which uses Euclidean ...
78 views

### Software library for Hidden Markov Modeling of a large text database

Given we have a large database of texts (e.g. product descriptions) and we want to extract multiple types of information (e.g. brand, release date, features, price, etc.) what's a good library to ...
315 views

### Understanding and applying sentiment analysis

I was just having been assigned a project of conducting sentiment analysis for some document collections. By Googling, a lot of sentiment-related research has popped up. My questions are: What are ...
310 views

### Similarity calculations for arrays

First of all, my apologies if I mess up the terminology. I've been out of math for several years, so I'm certain I'm going to use terms incorrectly. Also, though I concentrated mathematics in college, ...