# Questions tagged [cosine-similarity]

An angular-type similarity coefficient between two vectors. It is like correlation, only without centering the vectors.

114 questions
Filter by
Sorted by
Tagged with
0answers
11 views

### Should I consider the contribution of a variable in correspondance analysis if the $cos^2$ is weak?

So I have this table that represents contribution and cosinus of variable in CA I noticed that the most popular choice ( 1-5 hours ) has the weakest contributions and the weakest cosinus. It made ...
0answers
5 views

### Are absolute differences between factor scores a good way to quantify the similarity different things rated on the same factors?

I have x images rated on 25 dimensions. I also have x words rated on those 25 dimensions. When I factor analyze the ratings, I get the same four factors for images and words (though extracted in a ...
0answers
20 views

### How can I get the similarity and dissimilarity of two real numbers?

I would like to calculate the similarity and/or dissimilarity between two real numbers. The challenge is, we can have multiple types of real numbers. Say for example, Case-1: ...
0answers
36 views

### Cosine similarity with recentering - collaborative filtering

I don’t know much about stats (nor maths), so I’m sorry if I’m not being very clear on this... I’m trying to build a simple recommender system for books using collaborative filtering item by item. I’...
0answers
22 views

### Question related to using Pooled Output from BERT for similarity between sentences

I was hoping someone could give me advice and feedback on my current approach and possibly suggest to me a possible alternative. I'm trying to find the sentences that are most similar using the pooled ...
0answers
33 views

### Reconciling cosine similarity between vectors and subsets of these vectors

I'm seeing something that I'm having a hard time reconciling in my head. Essentially, the cosine similarity between two vectors I have is very low, but cosine similarities of their subsets are very ...
0answers
14 views

### How can i find similarity between two word vectors that aren't sentences?

I am looking to find similarity between multiple word vectors that are not exactly sentences. Imagine if each of the below rows starting from BodyPart1 is a vector. Example - Cow = [Tail,Legs,0,Moo] ...
0answers
13 views

### Is the word2vec embedding space an inner product space?

In their seminal paper Efficient Estimation of Word Representations in Vector Space (Mikolov et al. 2013) when performing vector arithmetics ...
0answers
49 views

### Finding Cosine similarity threshold (cut-off point) with Logistic Regression

I want to preprocess my data sets, removing similar articles. To do this, I'll perform cosine similarity, I don't know the best threshold (cut-off point) of my data set. I Found in this paper, there's ...
0answers
199 views

### Calculate similarity between two matrices

I have two matrices, $A$ and $B$, each of size $n\times m$, where $n$ is discrete time points, and $m$ are the variables measured (specifically, $n$ are dates and $m$ are investments measured in ...
1answer
49 views

### Baffled by cosine similarity - these results seem counterintuitive

Haven't used cosine similarity much in the past so getting into it now. Seeing results that are counterintuitive and would love your help making sense of them. Assume these simple vectors: ...
0answers
11 views

### Is cosine distance applicable for multiple classes?

Recently I learnt about DeepSort algorithm which use deep cosine metrics to re-identify objects. However, to my understadning, ...
0answers
12 views

### How to evaluate document similarity / content-based recommender system

I'm planning on building a basic content-based recommender system with word2vec and cosine similarity. The data consists of 300k documents in varying length. How do I evaluate my model if I have no ...
0answers
15 views

### How to improve facial recognition using cosine similarity

I'm using pretrained vgg16 model for feature extraction and then using cosine similarity to compare 2 embedding more like Siamese network. It gives descent results, above 60% for the true match and ...
0answers
43 views

### Euclidian distance vs cosine similarity

Currently I'm working on facial recognition. If I use encoding/feature vectors of 2 images which method will prove more accuracy, L2 norm or cosine similarity and why? I read "ICA performs ...
0answers
21 views

### logic behind two variables correlated at .99 but correlated differently to a third variable

Dataset has > 3000 observations. Each observation includes vectors A, B, and C. I compute the following new variables: Variable sim_B.C is coding the cosine similarity of vectors B and C Variable ...
0answers
14 views

### Metric for comparing cosine similarities to values in range(1, 5)

I am using cosine similarity as a metric for the semantic similarity of sentence pairs. ...
0answers
63 views

### What's the advantage of cosine distance over Jaccard distance for text document similarity

We usually use cosine distance as the similarity measure for text document. However, Jaccard distance also somehow make sense to me. My question is for text document, is there any advantage of using ...
1answer
104 views

### Cosine similarity for Categorical datasets?

Can I use Cosine similarity measure for estimating similarity/relationship between D1 and D2 (two categorical datasets)
0answers
23 views

### How to calculate similarity between two sets of items rated on a single dimension?

(I'm just making up variables for this example.) Let's say I have 100 words rated on their pleasantness. I also have 100 images rated on their pleasantness. I then had participants rate the fit ...
0answers
28 views

### cosine of angle between random variables is equal to the correlation coefficient? [duplicate]

I have seen this said multiple times where (1) the cosine of the angle between the random variables (on a vector space) is equal to the correlation coefficient, and (2) the claim if random variables ...
1answer
56 views

### Difference between an interaction and combining two variables into one with a similarity measure?

Conceptually, what would be the difference between interacting two variables in a regression and combining these two variables into one via a similarity measure? I'm looking at top managers and boards ...
0answers
140 views

### Performance of model decreases when calculating Euclidean distance between vectors with TF.IDF weigths compared to TF weights

My problem in short: I use Jaccard similarity, cosine similarity and euclidean distance to compute a similarity between documents. The documents consists of either the words,hashtags or combination of ...
0answers
31 views

### How to calculate the similarity of data with noise?

I'm stuck on calculating similarity. Please tell me in which direction to move. There are three files of different lengths that need to be compared for similarity. It is supposed to use the cosine ...
0answers
22 views

### What are the conditions for 2-dimensional curve fitting when the estimator function is a type of sine and cosine?

What are the conditions for 2-dimensional curve fitting when the estimator function is a type of sine and cosine? For example I wanted to estimate F=xy with Asin (((n*pi)/L)*x)sin ((((npi)/L)*z) ...
0answers
41 views

### Which text similarity algorithm should I use to compare the context of Instagram hashtags?

For a study I am comparing companies based on the posts written by their Instagram followers. I apply the following technique: Nike has 1.000.000 followers. 2000 random followers of Nike are selected ...
1answer
81 views

### Should we apply PCA before calculating similarities in high-dimensional space if my observations have length 1?

I have high-dimensional space (around 20 features) and I want to calculate similarity based on the angle of observation, not the magnitude. I have a nice function that can compute euclidean distance ...
0answers
80 views

### Normalized Levenshtein distance and triangle inequality

One question regarding to the triangle inequality of normalized Levenshtein Distance. I use the well-known form D(X,Y) = 1 - d(X,Y) / MAX(|X|,|Y|) where ...
1answer
103 views

### Rao-Stirling diversity index

I need help to calculate the Rao-Stirling diversity index. I tried it several times but cannot achieve equal results to R packages yet (e.g. diverse). I used the Rao-Stirling diversity index as ...
1answer
29 views

### How to find the list of nearest vectors if ony a vector is given?

I know there are many ways to compute similarity of two different non-zero vectors but is it possible to get a list of nearest vectors whose values are continous given a single continous vector. Lets ...
1answer
130 views

### Cosine similarity for recommendation systems

Recently picked up recommendation systems and was going through User Based Collaborative Filtering(UB-CF). Somewhere in the text, it specified that cosine similarity is one of the measures to find ...
1answer
121 views

### Why is the cosine similarity between these (seemingly uncorrelated) vectors so high?

I am calculating the correlations between vectors of experimental data by a variety of methods (Pearson's, cosine similarity, Euclidian distance, etc.). Most the results look fine, but occasionally ...
1answer
125 views

### Cosine similarity vs quadratic objective function

The aim is to calculate the similarity between two foods given the nutritional content of each. After some reading, it seems the most popular measure for this sort of problem is the cosine similarity ...
1answer
332 views

### Does it make sense to normalize vectors after PCA for cosine distance?

I start off with word2vec embeddings and process them in the following way: Standardize dimensions to mean 0 and standard deviation of 1 PCA to keep the top k-dimensional eigenvector, whereby ...
0answers
21 views

### Deep item-based recommender objective function

I'm trying to understand the following paper written by researchers at eBay that uses deep learning to overcome the problem of making recommendations when you mostly have one-of-a-kind items. A ...
0answers
215 views

### Extension of the K-Nearest neighbor algorithm to get results in different neighborhoods

I would like to use the kNN algorithm to find the closest neighbor to a vector. But I would like it to limit to a point per neighborhood (radius) In this image, given the point in red, I would like ...
0answers
347 views

### Should I normalize the vectors by row or by column before performing cosine similarity?

I have a dataset which contains vectors of different features that generated from subtitles in movies, something like: ...
0answers
159 views

### How can I define a accuracy measure for word2vec predictions

I have a data set consisting of tags and some classes.I'm suppose to find the nearest class to each set of tags with Word2vec embeddings and cosine similarity.Each set of tags have multiple classes ...
0answers
46 views

### the accuracy of covariance between two high-dimensional vectors

Question Is the covariance between high-dimensional vectors less accruate than covariance between two vectors in low-dimensional vecotrs? I am asking this questio to check if there is a need for '...
0answers
23 views

### analyze time-series consine similarity matrices (in R)

I have calculated the cosine similiarity between multiple users for 10 years separately. I can visualize and cluster the matrix of each year but I was wondering if there is a smart approach, vignette ...
0answers
93 views

### Are there advanced “cos similarity” that influence of dim size is less?

I found that the cosine similarity is affected to the effect of "Curse of dimension" by trying the following simulation. create(select) two vectors form uniform random numbers U[-1, 1], each dim = 2,...
1answer
23 views

### Did my text data come from two distinct distributions?

I have labeled text data from two different classes. I have calculated tfidf feature representation of all the sentences in question. I have a huge matrix where rows are sentences and columns are ...
0answers
156 views

### Set similarity as weight to ratings

I have a problem deciding which similarity function to use. I want to find the similarity between the users based on their requirements about computer performance metrics normalized to 1. Each user ...
0answers
42 views

### Cosine similarity matrix of linearly transformed inputs

Given a matrix $\mathbf{C}$ which contains pairwise cosine similarities between rows of a matrix $\mathbf{A}$, linearly transformed by matrix $\mathbf{U}$:  \mathbf{C} = K(\mathbf{UA}, \mathbf{UA}) \$...
1answer
86 views

### Distance metric with characteristics of cosine and Manhattan

I'm working on a project where I want to find similarities between groups of events. So far I have expressed groups of events as vectors of event counts and computing similarities between them. I'm ...
1answer
681 views

### Finding similar text - algorithms and evaluation

I've been asked to create a program that will rank similar texts to an input text given a collection of text. So far I've been using a tdidf representation and cosine similarity with a lot of regex-...
1answer
501 views

### Calculating similarity between two lists: high cosine similarity, but high RMSE

I want to see how similar two datasets are, as a way to justify that they can be used in similar contexts. In practice one dataset contains manually calculated data, and the other automatically ...
1answer
3k views

### Weighted Cosine Similarity

To convert cosine similarity to weighted cosine similarity, one can use at least two approaches. But I don't know which one is better. The first approach is to first reweight each vector and then ...
1answer
373 views

### Spectral Clustering of a skipgram model

I have a model where I'm applying Spectral Clustering to frequencies of words. My pipeline consists in TF-IDF, followed by a <...
1answer
217 views

### Text Similarity - Cosine - Control. Suggestion to another / better method?

I would like to ask you, if anybody could check my code, because it was behaving weird - not working, giving me errors to suddenly working without changing anything - the code will be at the bottom. ...