Linked Questions

156
votes
5answers
120k views

What's the difference between principal component analysis and multidimensional scaling?

How are PCA and classical MDS different? How about MDS versus nonmetric MDS? Is there a time when you would prefer one over the other? How do the interpretations differ?
126
votes
6answers
21k views

Is there an intuitive interpretation of $A^TA$ for a data matrix $A$?

For a given data matrix $A$ (with variables in columns and data points in rows), it seems like $A^TA$ plays an important role in statistics. For example, it is an important part of the analytical ...
47
votes
5answers
56k views

How do I test a nonlinear association?

For plot 1, I can test the association between x and y by doing a simple correlation. For plot 2, where the relationship is nonlinear yet there is a clear relation between x and y, how can I test the ...
19
votes
4answers
35k views

k-means implementation with custom distance matrix in input

Can anyone point me out a k-means implementation (it would be better if in matlab) that can take the distance matrix in input? The standard matlab implementation needs the observation matrix in input ...
28
votes
1answer
19k views

Converting similarity matrix to (euclidean) distance matrix

In Random forest algorithm, Breiman (author) constructs similarity matrix as follows: Send all learning examples down each tree in the forest If two examples land in the same leaf increment ...
15
votes
1answer
2k views

Is there an intuitive characterization of distance correlation?

I've been staring at the wikipedia page for distance correlation where it seems to be characterized by how it can be calculated. While I could do the calculations I struggle to get what distance ...
6
votes
2answers
5k views

Sums-of-Squares (total, between, within): how to compute them from a Distance Matrix?

I am having trouble understanding the concept of Sum of Squares in the context of distance matrices (Studer et al. 2010). The Sum of Squares I am familiar with is the classical $SS$ from ANOVA, ...
1
vote
2answers
2k views

Cosine-Similarity vs non-linear measures

In NLP, people often use cosine similarity to measure how close two vector spaces are to each other. However, we know that cosine-similarity is the same thing as Pearson correlation, for centered ...
1
vote
1answer
623 views

Why is distance covariance defined squared, while covariance is not?

I am dealing in a data science project with correlation analyses using pearson and distance correlation. While trying to understand the differences between them, I learned about the differences by ...
3
votes
0answers
695 views

Obtain within-group Gram matrix out of distance matrix

Gram matrix Let $\bf X$ be a n x p dataset with columns (variables) centered. Then p x p $\bf X'X$ is the total scatter matrix ...
3
votes
1answer
276 views

What is the distance correlation for Anscombe’s quartet?

Is the newer descriptive statistic (distance correlation) able to resolve these troubling four datasets (Anscombe’s quartet)?
2
votes
0answers
200 views

Does classic MultiDimensional Scaling reconstructs data upto a rotation when there's no noise and embedding dimensions equals original data dimension?

In the setup of classical MultiDimensional Scaling (MDS), assume that $D:=[d_{ij}]$ be an $n \times n$ distance matrix, i.e. $d(i,i)=0, d(i,j)=d(j,i) > 0 \forall i, j = 1 \dots n.$ Assume that: ...
1
vote
0answers
63 views

Secret Life of Covariance Matrix

I am currently reading Secret Life of Covariance Matrix: http://www.inf.fu-berlin.de/inst/ag-ki/rojas_home/documents/tutorials/secretcovariance.pdf and am confused by the following: Now, in the ...