2
$\begingroup$

There are related questions being asked already but my problem is i can't find a good method of measuring similarity between two datasets that are represented by various lengths of matrices. For instance, first dataset is a sensor data with x,y,z,gyro,acc features of 1000 records. The second dataset's features are the same but with 1500 records. So how do I compute the similarity between these two.

I've used dynamic time warping (DTW) but not sure about it because it is mostly used for time-based operations but my dataset doesn't contain any temporal info. Also, it doesn't output a score between [0-1], so not sure how to scale it. I checked Kolmogorov-Smirnov Test, as well, but it can give me the difference between only a particular feature (column) of different size. I thought of measuring the distance for each column separately and summing it up, but haven't tried yet.

$\endgroup$
  • 1
    $\begingroup$ What does "similarity" mean to you in your situation? $\endgroup$ – gung May 30 '16 at 19:38
  • $\begingroup$ I want to know whether the datasets were generated by the same user or not $\endgroup$ – Hayro May 30 '16 at 19:40
  • $\begingroup$ Would it be enough to see if the variables had similar means, etc? $\endgroup$ – gung May 30 '16 at 19:42
  • $\begingroup$ There are lot of users, and each user has many features. I need a more robust way because I'll run an algorithm on the users whose data change at different sessions. $\endgroup$ – Hayro May 30 '16 at 19:46
  • $\begingroup$ If these are time series data, and you believe same user will generate similar time series, you can use cross-correlation as a measure of similarity $\endgroup$ – A. Ray May 31 '16 at 1:48
1
$\begingroup$

I think your problem is related to domain adaptation and the term "discrepancy" they use in the literature between two domains. Check Ben David papers on discrepancy. There is a simplified measure defined as the classification error (or accuracy) of a classifier that discriminates samples of one dataset from the other.

$\endgroup$
  • $\begingroup$ Can you expand a bit your answer? $\endgroup$ – Joe_74 May 29 '17 at 14:38
  • 1
    $\begingroup$ Lets say you have datasets A and B. You build the new dataset X composed of both A and B with labels a if the data belongs to A and b if data belongs to B. Then you train a classifier on the new dataset that discriminates a from b. you measure abs(2*acc-1) where acc is the accuracy between 0 and 1. If the datasets are similar, you will get 0.5 accuracy for two class classification which is the minimum of the discrepancy defined above. otherwise it will return positive nonzero discrepancy. $\endgroup$ – iBM May 30 '17 at 15:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.