Linked Questions

2
votes
0answers
1k views

Convert Pearson correlation matrix into dissimilarity matrix [duplicate]

I would like to execute multidimensional scaling (MDS) based on a matrix of Pearson correlation coefficients. The sklearn.manifold.MDS function takes a dissimilarity matrix as an input and I therefore ...
0
votes
1answer
181 views

how a cosine similarity measure can be made a distance measure [duplicate]

I want to make cosine similarity a distance measure for undersampling of fraud data , but the problem is if I get the zero values it will give 0/0, what can I do for this what can be the ...
111
votes
6answers
14k views

Is there an intuitive interpretation of $A^TA$ for a data matrix $A$?

For a given data matrix $A$ (with variables in columns and data points in rows), it seems like $A^TA$ plays an important role in statistics. For example, it is an important part of the analytical ...
65
votes
6answers
94k views

Why does k-means clustering algorithm use only Euclidean distance metric?

Is there a specific purpose in terms of efficiency or functionality why the k-means algorithm does not use for example cosine (dis)similarity as a distance metric, but can only use the Euclidean norm? ...
36
votes
3answers
40k views

Why does correlation matrix need to be positive semi-definite and what does it mean to be or not to be positive semi-definite?

I have been researching the meaning of positive semi-definite property of correlation or covariance matrices. I am looking for any information on Definition of positive semi-definiteness; Its ...
18
votes
9answers
9k views

Pairwise Mahalanobis distances

I need to calculate the sample Mahalanobis distance in R between every pair of observations in a $n \times p$ matrix of covariates. I need a solution that is efficient, i.e. only $n(n-1)/2$ distances ...
28
votes
2answers
25k views

Is cosine similarity identical to l2-normalized euclidean distance?

Identical meaning, that it will produce identical results for a similarity ranking between a vector u and a set of vectors V. I have a vector space model which has distance measure (euclidean ...
14
votes
4answers
18k views

Prove the equivalence of the following two formulas for Spearman correlation

From wikipedia, Spearman's rank correlation is calculated by converting variables $X_i$ and $Y_i$ into ranked variables $x_i$ and $y_i$, and then calculating Pearson's correlation between the ranked ...
10
votes
3answers
31k views

Which distance to use? e.g., manhattan, euclidean, Bray-Curtis, etc

I am not a community ecologist, but these days I am working on community ecology data. What I couldn't understand, apart from the mathematics of these distances, is the criteria for each distance to ...
17
votes
1answer
15k views

What is the proper association measure of a variable with a PCA component (on a biplot / loading plot)?

I am using FactoMineR to reduce my data set of measurements to the latent variables. The variable map above is clear for me to interpret, but I am confused when ...
13
votes
3answers
28k views

Euclidean distance score and similarity

I'm just working with the book Collective Intelligence (by Toby Segaran) and came across the Euclidean distance score. In the book the author shows how to calculate the similarity between two ...
15
votes
2answers
8k views

Understanding distance correlation computations

As far as I understood, distance correlation is a robust and universal way to check if there is a relation between two numeric variables. For example, if we have a set of pairs of numbers: ...
22
votes
1answer
26k views

Using correlation as distance metric (for hierarchical clustering)

I would like to hierarchically cluster my data, but rather than using Euclidean distance, I'd like to use correlation. Also, since the correlation coefficient ranges from -1 to 1, with both -1 and 1 ...
13
votes
4answers
5k views

Is triangle inequality fulfilled for these correlation-based distances?

For hierarchical clustering I often see the following two "metrics" (they aren't exactly speaking) for measuring the distance between two random variables $X$ and $Y$: $\newcommand{\Cor}{\mathrm{Cor}}$...
11
votes
1answer
10k views

Arrows of underlying variables in PCA biplot in R

At the risk of making the question software-specific, and with the excuse of its ubiquity and idiosyncrasies, I want to ask about the function biplot() in R, and, ...

15 30 50 per page