Questions tagged [cosine-similarity]

An angular-type similarity coefficient between two vectors. It is like correlation, only without centering the vectors.

Filter by
Sorted by
Tagged with
0 votes
0 answers
18 views

LSA and cosine similarity approximation with large matrix

This question is the non-r related version of this one I have a tdm with 16k x 350k dimension, for which I am trying to get the document-document similarity (cosine type). With RSpectra I have found a ...
user avatar
0 votes
0 answers
27 views

How to calculate Cosine Similarity from Keras model?

I'm trying to make hybrid recommender system that recommends movies to users from Movielens dataset. Its Content part is based on Doc2Vec model from gensim library and its Collaborative Filtering part ...
user avatar
0 votes
0 answers
8 views

Understanding distance correlation computation for Multivariate Data

The original question and answer is in this link : Understanding distance correlation computations Since I do not have enough reputation, I don't have the right to comment. This is the main reason why ...
user avatar
0 votes
0 answers
46 views

Scalar similarity measure between two vectors including both angle and magnitude

I have different models, predicting a vector $\boldsymbol{v}\in\mathbb{R}^3$. Now I would like to compare the performance of these models against a baseline vector $\boldsymbol{b}\in\mathbb{R}^3$, for ...
user avatar
  • 251
0 votes
0 answers
24 views

What is the difference in interpretatation between Cosine Similarity and Correlation? [duplicate]

I'm studying PCA analysis and working with the FactoMineR an factoextra libraries in R. After generating a PCA object (with the PCA() or prcomp() functions) I access the "var" variable to ...
user avatar
  • 203
0 votes
0 answers
27 views

Given embedding vector A and vector B, how to find top k embedding vectors such that they are similar to vector A and dissimilar to vector B

Which would be better approach for getting top k embedding vectors such that they are similar to embedding vector A and dissimilar to vector B. Approach 1: calculate ...
user avatar
2 votes
1 answer
167 views

Cosine similarity seems to perform better with higher dimensions than Euclidean distance? Should this be the case?

I've generated 100 random vectors (data points) in n∈[1,...,50] dimensions. I then compared distances between each pair of vectors and calculated the mean value. I've done this for all dimensions ...
user avatar
0 votes
0 answers
21 views

How Can I Reduce Similarity Analysis of Multiple Time-Series Vectors into a Single Value?

I have ~15 independently-sourced vectors with about 1600 samples in each. They are basically continuous, ~1 Hz, from t=0 to 22 minutes. The nature of the dataset is such that the signals are ...
user avatar
0 votes
1 answer
39 views

Search, rank and recommend in large text datasets

Imagine you are Spotify and you have billions of songs. Assume that each of these songs are transcribed into text. How do you design your search and recommendation pipeline such that when somebody ...
user avatar
  • 118
0 votes
0 answers
13 views

Why very different cosine similarity scores when: mean(cos_sim(A, B), (A, C)) than when comparing to concatenated: cos_sim(A, concat(B, C))

I am trying to compare rhetorical similarity of politicians to the rest of their party in 2010. Say party 1 consists of politician A, B and C. Each politician has made one speech in 2010. If I want to ...
user avatar
  • 1
0 votes
0 answers
26 views

Why is cosine similarity a lot higher between one document and all other documents grouped than vs the mean of each

I am trying to calculate a cosine similarity score for speeches made by various politicians. Say I have politician A who belongs to party A. Party A has 20 other politicians who have made a total of ...
user avatar
  • 1
0 votes
0 answers
19 views

Measuring similarity between students to build an exam recommender system

I am playing around with the following problem: I have various groups of students from various schools and they receive online questions. A group receives the same questions, but different groups in ...
user avatar
2 votes
1 answer
141 views

cosine similarity with weights

We're doing pairwise similarity computation for some real estate properties. Our data goes something like this: ...
user avatar
  • 490
0 votes
0 answers
138 views

Is cosine similarity applicable for BM25 weighted vectors?

It seems to be very common to use cosine similarity for TF-IDF weighted vectors for comparison of documents and their retrieval. I am currently looking into the BM25 weighting scheme and here it seems ...
user avatar
0 votes
0 answers
16 views

How to calculate the level of cosine similarities across observations within a class and compare with other classes in text analysis?

I know that I can use cosine similarity to calculate the pairwise similarity between any two observations. But my problem is that I want to see if the observations in one class is more similar to each ...
user avatar
0 votes
0 answers
33 views

measure the overall similarity between two vectors

I have two paired vectors which report the number of times each object was found in the two experiments (the count value could be 0 or greater (e.g. 5,10,1000, etc)). I want to compare them and get a ...
user avatar
1 vote
0 answers
149 views

Cosine similarity with vectors of different lengths (time series)

I have time series of different lengths with different patterns. I want to find the similarity between them in such a way that it is defined from 0 to 1 and the measure takes into account the length (...
user avatar
  • 23
0 votes
0 answers
21 views

Is combination of squared Euclidean and normalized cosine distance follow Bregman-divergence?

I know that squared euclidean distance satisfy the property of Bregman-divergence. I wanted to do some experiments using combination of various distance metric. So I am curious to know if I add ...
user avatar
0 votes
0 answers
15 views

how to calculate cosine similarity for bow

let's say i have these two sentences: S1:premier league meet target audience watch game buy messi S2:everyone offer discount ticket watch messi these two have went through stopword removal, ...
user avatar
0 votes
0 answers
36 views

Searching for a distance metric or distance similarity to compare two arrays of counts

Every object in my dataset is described as a vector of n = 200 continuous counts. I need to choose a measure to evaluate the similarity of two objects in the dataset. I want to satisfy the following ...
user avatar
  • 3
1 vote
0 answers
18 views

Should I consider the contribution of a variable in correspondance analysis if the $cos^2$ is weak?

So I have this table that represents contribution and cosinus of variable in CA I noticed that the most popular choice ( 1-5 hours ) has the weakest contributions and the weakest cosinus. It made ...
user avatar
  • 187
0 votes
0 answers
9 views

Are absolute differences between factor scores a good way to quantify the similarity different things rated on the same factors?

I have x images rated on 25 dimensions. I also have x words rated on those 25 dimensions. When I factor analyze the ratings, I get the same four factors for images and words (though extracted in a ...
user avatar
  • 1,641
1 vote
0 answers
49 views

Cosine similarity with recentering - collaborative filtering

I don’t know much about stats (nor maths), so I’m sorry if I’m not being very clear on this... I’m trying to build a simple recommender system for books using collaborative filtering item by item. I’...
user avatar
2 votes
0 answers
96 views

Question related to using Pooled Output from BERT for similarity between sentences

I was hoping someone could give me advice and feedback on my current approach and possibly suggest to me a possible alternative. I'm trying to find the sentences that are most similar using the pooled ...
user avatar
  • 21
1 vote
0 answers
269 views

Finding Cosine similarity threshold (cut-off point) with Logistic Regression

I want to preprocess my data sets, removing similar articles. To do this, I'll perform cosine similarity, I don't know the best threshold (cut-off point) of my data set. I Found in this paper, there's ...
user avatar
0 votes
0 answers
618 views

Calculate similarity between two matrices

I have two matrices, $A$ and $B$, each of size $n\times m$, where $n$ is discrete time points, and $m$ are the variables measured (specifically, $n$ are dates and $m$ are investments measured in ...
user avatar
  • 233
1 vote
1 answer
86 views

Baffled by cosine similarity - these results seem counterintuitive

Haven't used cosine similarity much in the past so getting into it now. Seeing results that are counterintuitive and would love your help making sense of them. Assume these simple vectors: ...
user avatar
  • 163
1 vote
0 answers
197 views

Euclidian distance vs cosine similarity

Currently I'm working on facial recognition. If I use encoding/feature vectors of 2 images which method will prove more accuracy, L2 norm or cosine similarity and why? I read "ICA performs ...
user avatar
2 votes
1 answer
153 views

Cosine similarity for Categorical datasets?

Can I use Cosine similarity measure for estimating similarity/relationship between D1 and D2 (two categorical datasets)
user avatar
  • 41
0 votes
0 answers
40 views

How to calculate similarity between two sets of items rated on a single dimension?

(I'm just making up variables for this example.) Let's say I have 100 words rated on their pleasantness. I also have 100 images rated on their pleasantness. I then had participants rate the fit ...
user avatar
  • 1,641
1 vote
0 answers
30 views

cosine of angle between random variables is equal to the correlation coefficient? [duplicate]

I have seen this said multiple times where (1) the cosine of the angle between the random variables (on a vector space) is equal to the correlation coefficient, and (2) the claim if random variables ...
user avatar
2 votes
1 answer
164 views

Difference between an interaction and combining two variables into one with a similarity measure?

Conceptually, what would be the difference between interacting two variables in a regression and combining these two variables into one via a similarity measure? I'm looking at top managers and boards ...
user avatar
0 votes
0 answers
25 views

What are the conditions for 2-dimensional curve fitting when the estimator function is a type of sine and cosine?

What are the conditions for 2-dimensional curve fitting when the estimator function is a type of sine and cosine? For example I wanted to estimate F=xy with Asin (((n*pi)/L)*x)sin ((((npi)/L)*z) ...
user avatar
  • 1
1 vote
0 answers
87 views

Which text similarity algorithm should I use to compare the context of Instagram hashtags?

For a study I am comparing companies based on the posts written by their Instagram followers. I apply the following technique: Nike has 1.000.000 followers. 2000 random followers of Nike are selected ...
user avatar
  • 21
2 votes
1 answer
234 views

Should we apply PCA before calculating similarities in high-dimensional space if my observations have length 1?

I have high-dimensional space (around 20 features) and I want to calculate similarity based on the angle of observation, not the magnitude. I have a nice function that can compute euclidean distance ...
user avatar
  • 185
3 votes
0 answers
150 views

Normalized Levenshtein distance and triangle inequality

One question regarding to the triangle inequality of normalized Levenshtein Distance. I use the well-known form D(X,Y) = 1 - d(X,Y) / MAX(|X|,|Y|) where ...
user avatar
  • 31
1 vote
1 answer
162 views

Rao-Stirling diversity index

I need help to calculate the Rao-Stirling diversity index. I tried it several times but cannot achieve equal results to R packages yet (e.g. diverse). I used the Rao-Stirling diversity index as ...
user avatar
  • 11
1 vote
1 answer
76 views

How to find the list of nearest vectors if ony a vector is given?

I know there are many ways to compute similarity of two different non-zero vectors but is it possible to get a list of nearest vectors whose values are continous given a single continous vector. Lets ...
user avatar
  • 2,031
1 vote
1 answer
454 views

Cosine similarity for recommendation systems

Recently picked up recommendation systems and was going through User Based Collaborative Filtering(UB-CF). Somewhere in the text, it specified that cosine similarity is one of the measures to find ...
user avatar
0 votes
1 answer
296 views

Why is the cosine similarity between these (seemingly uncorrelated) vectors so high?

I am calculating the correlations between vectors of experimental data by a variety of methods (Pearson's, cosine similarity, Euclidian distance, etc.). Most the results look fine, but occasionally ...
user avatar
  • 321
0 votes
1 answer
178 views

Cosine similarity vs quadratic objective function

The aim is to calculate the similarity between two foods given the nutritional content of each. After some reading, it seems the most popular measure for this sort of problem is the cosine similarity ...
user avatar
0 votes
1 answer
565 views

Does it make sense to normalize vectors after PCA for cosine distance?

I start off with word2vec embeddings and process them in the following way: Standardize dimensions to mean 0 and standard deviation of 1 PCA to keep the top k-dimensional eigenvector, whereby ...
user avatar
1 vote
0 answers
24 views

Deep item-based recommender objective function

I'm trying to understand the following paper written by researchers at eBay that uses deep learning to overcome the problem of making recommendations when you mostly have one-of-a-kind items. A ...
user avatar
  • 6,017
0 votes
0 answers
256 views

Extension of the K-Nearest neighbor algorithm to get results in different neighborhoods

I would like to use the kNN algorithm to find the closest neighbor to a vector. But I would like it to limit to a point per neighborhood (radius) In this image, given the point in red, I would like ...
user avatar
1 vote
0 answers
563 views

Should I normalize the vectors by row or by column before performing cosine similarity?

I have a dataset which contains vectors of different features that generated from subtitles in movies, something like: ...
user avatar
  • 41
1 vote
0 answers
193 views

How can I define a accuracy measure for word2vec predictions

I have a data set consisting of tags and some classes.I'm suppose to find the nearest class to each set of tags with Word2vec embeddings and cosine similarity.Each set of tags have multiple classes ...
user avatar
1 vote
0 answers
64 views

the accuracy of covariance between two high-dimensional vectors

Question Is the covariance between high-dimensional vectors less accruate than covariance between two vectors in low-dimensional vecotrs? I am asking this questio to check if there is a need for '...
user avatar
0 votes
0 answers
23 views

analyze time-series consine similarity matrices (in R)

I have calculated the cosine similiarity between multiple users for 10 years separately. I can visualize and cluster the matrix of each year but I was wondering if there is a smart approach, vignette ...
user avatar
2 votes
0 answers
145 views

Are there advanced "cos similarity" that influence of dim size is less?

I found that the cosine similarity is affected to the effect of "Curse of dimension" by trying the following simulation. create(select) two vectors form uniform random numbers U[-1, 1], each dim = 2,...
user avatar
  • 53
0 votes
1 answer
24 views

Did my text data come from two distinct distributions?

I have labeled text data from two different classes. I have calculated tfidf feature representation of all the sentences in question. I have a huge matrix where rows are sentences and columns are ...
user avatar