3
$\begingroup$

I have a ground truth list, call it A, and I have different methods for generating lists that should be somewhat similar.. I want to decide which method of these generates a list that is most similar to the ground truth.. The list consists of strings..

I am new to these stuff, I tried average precision and the results made sense, then I was trying to use Spearman rank correlation, but I ended up comparing something like A = ["abd", "abc", "cad"] and B = ["abd", "decf", "hhj", "cad", .. ] so the list sizes were different, is it even possible to use Spearman here?

And what other methods can I use?

$\endgroup$
1
  • 1
    $\begingroup$ What are the important features of your lists? (e.g. ordering, size, etc.). For the basic case where you'd like to measure the degree of overlap between 2 lists, something like Jaccard's Similarity would make sense. $\endgroup$
    – Nick
    Nov 3, 2011 at 17:41

1 Answer 1

5
$\begingroup$

You could use the following procedure to compare lists of strings:

  1. First, define a distance metric that measures how close any two strings are. See the wiki for several string metrics that you can use.

  2. Second, compute the distance of each element of your candidate list with that of the truth list. So, for A you would compute the distance between the first pair "abc" and its corresponding truth value; the second pair "decf" and its corresponding truth value and so on. The result is a distance vector that gives the distance between each string in A and its corresponding true values.

  3. Third, you need some measure to compute which list is closer to the true list. A useful metric by analogy to ordinary least squares would be the sum of squared distances for each vector.

You then chose the candidate list that has the lowest overall sum from point 3 above as the 'closest' list to your truth value.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.