Measures of closeness between distributions, clusterings, data sets or other objects.

learn more… | top users | synonyms

0
votes
0answers
4 views

Similarity measure for clusterings in graphs evolving over time (temporal network)

From what I understand about clusters, they can be obtained from an existing graph at 1 instance of time. But consider the situation of a temporal network, such as a social network, where the graph ...
-3
votes
0answers
31 views

Brainerd-Robinson similarity test or alternative

Is there an R package that provides a Brainerd-Robinson test for similarity? I work in archaeology, use R almost exclusively for our quantitative work, and BR is a very common approach for assemblage ...
0
votes
0answers
11 views

Similarity measure between groups of monetary values

Suppose I have 2 groups, with monetary values, like this: How could I get a measurement of similarity between these 2 groups? It could be just a hint for me to get started with, like a name of the ...
1
vote
0answers
41 views

Properties of Levenshtein, N-Gram, cosine and Jaccard distance coefficients - in sentence matching

Let's say I have two strings: string A: 'I went to the cafeteria and bought a sandwich.' string B: 'I heard the cafeteria is serving roast-beef sandwiches today'. ...
2
votes
3answers
35 views

pattern recognition for sequence data

There are a lot of data sequences, I am trying to find pairs of sequences that are similar with other. Trivially, we can define some distance measure, and compare each pair of sequence in terms of ...
1
vote
0answers
16 views

How to preprocess features for supervised similarity learning

I have a data set of profiles that describes a set of people and a binary classification describing when some of these people have been grouped together in the past which I am assuming implies ...
5
votes
1answer
153 views

Similarity function with given properties

I would like to find a similarity function $f$ between two values (each value is continuous and is bounded by $[0,1]$) that would have the following properties: $$ f(1, 1) = 0.5 $$ $$ f(0.5, 0.5) =...
0
votes
0answers
45 views

how to measure similarity of two datasets (matrices) of different length

There are related questions being asked already but my problem is i can't find a good method of measuring similarity between two datasets that are represented by various lengths of matrices. For ...
1
vote
1answer
20 views

How to evaluate level of significance of two similar correlations?

I have four datasets: A1, A2, B1, B2. Every dataset has between 100-300 items. Every item in every dataset has two values: x, y The goal: Find what datasets have similar x values. If the datasets ...
0
votes
1answer
25 views

Assessing correlation/similarity in shape and position of two datasets

I have data-sets within which position and shape (curves, relative increase/decrease at any given x-value) are the defining characteristics, such as: And a simulation which attempts to recapture ...
1
vote
2answers
48 views

What is maximum and its computation in the function dist() {stats} in R?

In R, we can calculate a distance matrix using the method "maximum" in the function dist() in the stats package. ...
0
votes
0answers
15 views

Using frequency tables to judge similarity

I'm a newbie to data mining, and attempting to simplify a problem. Assume I want to find out what skills make up a certain profession. So I'll ask the user to input their profession, followed by a ...
3
votes
1answer
76 views

Finding most similar observations in two data sets

I have the following question. Given are two data sets A and B with around 10,000 observations described by different variables categorical and numeric. The task is to find these observations in B ...
0
votes
1answer
68 views

distance measure of two discrete probability histograms (distance between two vectors)

I have multiple sets of discrete probability histograms(vectors) and I want to measure the distance between each histogram. I have done some research but I am in doubt. Literature suggest I could ...
0
votes
0answers
24 views

Numerical Similarity Measures for Dirty Data

I need to compare two datasets to see if they are "similar" in terms of numerical similarity between given columns. The problem is, however, that I do not know the characteristics of the 2nd dataset. ...
0
votes
1answer
23 views

How to gauge and specify similarity of energy data for two US states?

I am working on a research project to gauge the potential for rooftop photo-voltaic solar power in Wisconsin in the years 2020 and 2030. Part of this research involves analyzing hourly electric usage ...
-1
votes
1answer
46 views

proper Similarity measure and clustering algorithm for binary data

data sample as follow : interest to find clusters of similar users, pages number around 100 pages. users around 1000 , i would like to know what are proper Similarity/dis. measure can used in this ...
1
vote
0answers
85 views

Outlier similarity

I have a group of sensors where each sensor records 10 different binary data points which is then saved as an integer that can range from 0 to 2^10 - 1. To make sure that's clear, to put it another ...
0
votes
0answers
28 views

Clustering subjects regarding binary property vectors

My data set consists of 120 subjects and 50 binary attributes. I want to find clusters in that. So far I started by a visual analysis plotting the n x n matrix taking the several similarity measures (...
0
votes
1answer
54 views

How to soften or mitigate vector similarity measure?

I would like to evaluate a similarity between two objects X and Y by comparing a neighbourhood in which they're located. I construct two sets of nine concentric and equidistant circles with centers ...
1
vote
1answer
34 views

Selecting the number of hashes for minhash? Working with extremely sparse data and want more collisions

I'm attempting to use minhash to generate clusters and similarities, and I am primarily using ideas from these resources. http://www2007.org/papers/paper570.pdf https://chrisjmccormick.wordpress.com/...
0
votes
0answers
23 views

Recommender system - combining collaborative filtering and content based

I read a lot of stuffs about combining collaborative filtering and content-based but no one seems to bother explaining how to get those so-called hybrid recommenders or they do it so shallowly that I ...
0
votes
0answers
18 views

Is spatial autocorrelation and similarity the same?

I am new to autocorrelation analysis. I want to test if my sampling points are different when the distance increase between them. So here is my question: When there is no spatial autocorrelation does ...
1
vote
0answers
18 views

Fixed-length time series difference measurement

I have a very specific case of time series comparison. Suppose we have two time series $T_c$ and $T_s$, both of them $512$ discrete points in size. Both of them have been sampled at the same ...
0
votes
0answers
50 views

Calculate similarity of probability vector and TFIDF vector

I want to compute the similarity between documents of two groups of documents. All documents have the same term vector of length 100, but documents in group A are probabilities, whereas documents in ...
1
vote
2answers
94 views

how to measure similarity of two lists of continuous data with different length

I have two lists of continuous data with different length. a) How should I measure (dis)similarity of these two lists? or, as these lists can be formed into histogram, how can I quantify (dis)...
1
vote
0answers
34 views

Cluster analysis of large, multiple answer dataset

I have to analyse data from a marketing study. I will use SPSS. The questionnaire will look like this: Q: Imagine Situation X. Select 1-3 Criteria from the list that best describe your feeling. ...
1
vote
2answers
69 views

Classifying time-series similarity - what variable should I train on?

I have ~10,000 time series, each with 65 time points. I'm interested in classifying each pair of time series as "similar" or "not similar". Here's an example of two similar (left) and not similar time ...
1
vote
0answers
63 views

Similarity between sets with different size

is there a distance measure like jaccard for sets with different sizes? For example A=['a','b','c'] and B=['a','d'] I would like to include the total intersection as well as the order. The ...
1
vote
1answer
69 views

How to find similarity matrix from a confusion matrix?

How to find similarity matrix from a confusion matrix of a multiclass classification? For example in this paper: Exploiting confusion matrices for automatic generation of topic hierarchies and ...
0
votes
0answers
18 views

To what degree does Mahalanobis distance account for correlations of the data?

I understand that Mahalanobis distance is used when your data are correlated (e.g., as with many environmental variables), but just how correlated can the variables be? Should I be screening and ...
0
votes
1answer
35 views

Error when computing Cosine Similarity

I'm trying to compute the cosine similarity of the following vector: 1 1 0 1 2 0 1 1 $ \frac {1*2+1*0+0*1+1*1}{ \sqrt(1^2+1^2+0^2+1^2)\sqrt(2^2+0^2+1^2+1^2)}$ ...
0
votes
1answer
78 views

Order a correlogram

I have 30 days data, and I am plotting correlogram using corrplot (R package). I have an option to order the correlogram with hclust option and hence I obtain plot as shown on right side of below ...
1
vote
0answers
18 views

What does it mean when least squares and instrumental variables give similar results?

What can said when for some data set the result of a least squares regression is extremely close to the result of an instrumental variable regression?
0
votes
1answer
100 views

Alternate distance metrics for two time series

I have time-series data of different houses. Assume it is power consumption data. Now, I want to cluster the houses following similar power consumption pattern utmost. So, the various distance metrics ...
0
votes
0answers
23 views

Similarity measure for short series

I am looking for the best similarity measure between pairs of equal length series. These are concentrations of molecules measured in discrete fractions, the number of fractions is small (5-6). The ...
0
votes
1answer
63 views

Preprocessing survey data for clustering

I want to find 4-10 clusters in survey data with 100 questions answered by 2000 individuals using a technique such as K-means or Gaussian Mixture Models. There is no response variable so the ...
3
votes
1answer
354 views

Implicit feature space of Power Kernel

For the polynomial kernel, $K(x,y) = (x^Ty+c)^d$, the implicit feature space $\phi$ for which $K(x,y) = \phi(x)^T \phi(y)$ is of finite dimension and well known [1][2]. It is also well known that the ...
0
votes
0answers
15 views

How to find similar documents with mixed features

I am working on finding a similar documents problem. I have 3 sets of features ...
0
votes
1answer
67 views

How to find similar document with new vocabulary

I am working on a problem of finding similar documents. I am using a Tf-Idf based vector space model representation of documents and it gives me good results. However when I encounter a document ...
0
votes
0answers
27 views

Multivariate DTW

I am not able to find any resources in case of DTW for multivariate signals. For example, I would like to find DTW similarity between two objects where each object contains following signals: ...
2
votes
2answers
62 views

Proper statistical analysis for similarity of two paired datasets

I am an engineering student and as part of an undergraduate research, I created a device that measures a certain value. However, there is a commercial device that measures that same value. Here's ...
0
votes
2answers
43 views

Cluster users based on file access and application usage

I have access to file access data for employees in my organisation. For each employee I can see what read/write operations they carried out in the last month or so. Per operation, I see a user ID, ...
0
votes
0answers
23 views

Cluster logistic regression datasets

Let's say that I've got a 100 advertising campaigns and for each I'm trying to predict that a user will click on the advertisement or not using logistic regression. So basically I'm creating 100 ...
1
vote
0answers
18 views

Similarity between an item and a set

I have two items $I_0$ and $I_1$ and a set $S$. What I want to achieve is to compare which item (either $I_0$ or $I_1$) is more similar to the set. I can compare both $I_0$ and $I_1$ to every element ...
0
votes
0answers
13 views

Modelling distributions evolving with time

Say we have N probability vectors of dimensionality D describing how a process evolves in N time steps, something like multidimensional time series. What's the best way to model this sequence in ...
1
vote
1answer
81 views

Similarity measures between bimodal distributions

I have to compare two bimodal distributions. So far, I have found the Bimodality Coefficient: $$ BC = \frac{\mu_{3}^{2} + 1}{\mu_{4} + 3\frac{(n-1)^2}{(n-2)(n-3)}}, $$ with $\mu_{3}$ referring to ...
5
votes
1answer
76 views

What is the purpose of row normalization

I understand the reasoning behind column normalization, as it causes features to be weighted equally, even if they are not measured on the same scale - however, often in the nearest neighbour ...
1
vote
1answer
149 views

Similarity / clustering methods for temporal event data

I am looking for methods to calculate a reasonable similarity metric or clustering for my data. I have data in the form of a number of distinct events (lets call them A, B, C, D, ...), and the time ...
0
votes
0answers
43 views

should I count repeated items in jaccard similarity?

I have two lists: A and B where Size(A) = 4 and size(B) = 10. I would like to find the Jaccard similarity between these two lists. Suppose A = {Tom, George, John, Jennifer} and B={Tom, Jessica, ...