0
$\begingroup$

Given two (or maybe more) datasets with the same samples/members, but with different variables. Is there a general way to compare the information available in the two datasets without looking into the details of the variables?

So for example:

dataset1:

        | var1 | var2 | var3 | var4 | var5 | var6 |
sample1 |  7   |  4   |  5   |  3   |  3   |  4   |
sample2 |  9   |  4   |  5   |  4   |  8   |  2   |
sample3 |  7   |  5   |  1   |  9   |  4   |  3   |

dataset2:

        | var7 | var8 | var9 | var10| var11| var12|
sample1 |  3   |  7   |  1   |  2   |  8   |  7   |
sample2 |  6   |  3   |  3   |  6   |  8   |  4   |
sample3 |  3   |  1   |  4   |  3   |  2   |  6   |

How similar are these datasets?

$\endgroup$

1 Answer 1

0
$\begingroup$

So you want to know if any columns in the first data set exist disguised in the second one? How do you want to define similarity? You could start by standardizing both tables (subtract the column means and divide by the L^2 norm) then take the inner products of the matrices. The large elements in the resulting square, covariance matrix would correspond to the similar variables.

$\endgroup$
2
  • $\begingroup$ I am not necessarily looking in a column-per-column comparison. For example it could be that var7 = var1 + var2 and var8 = var1 - var2. Also, I am not really sure how to define similarity. Maybe we can use the distances (norm) between the samples in the scaled matrices as some kind of measure? $\endgroup$
    – user347583
    Commented Nov 22, 2014 at 21:54
  • $\begingroup$ How do you want to define similarity in that case? You appear to have an over-complete basis, so you will always be able to express the variables in the second dataset in terms of the those in the first. $\endgroup$
    – Emre
    Commented Nov 22, 2014 at 22:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.