3
$\begingroup$

I have two strings, e.g

  • str1="abddbabc" and
  • str2="bbcadbbd".

I know that each letter is representative of a floating point number, but I don't know what that number is. The only information that I have is that if a letter has higher alphabetical order its floating point value is larger (e.g floating point value of b is greater than a).

Is there anyway to compute Pearson's correlation (or any other association) between two string by knowing this information?

Improve this question
$\endgroup$
4
  • 3
    $\begingroup$ If you only know the rank order, then you can only compute rank correlation $\endgroup$
    – kjetil b halvorsen ♦
    Commented Oct 7, 2017 at 15:56
  • $\begingroup$ @kjetilbhalvorsen What if we add an assumption that the by increasing order, floating point assigned to the next letter increases by a fixed value, e.g floating_point(b)- floating_point(a)=floating_point(c)- floating_point(b) and we know the difference e.g 0.08 $\endgroup$
    – starrr
    Commented Oct 7, 2017 at 16:05
  • 4
    $\begingroup$ Since correlation doesn't change when values are shifted or rescaled, simply assign the values a=1, b=2, c=3, etc., and compute the correlation accordingly. It will be exactly the same as if you had the original values. But are you confident in that assumption? If not, then follow @Kjetil's suggestion. $\endgroup$
    – whuber ♦
    Commented Oct 7, 2017 at 16:21
  • 1
    $\begingroup$ @starrr if you want to add conditions on the question, please make sure to also edit them into the question itself $\endgroup$
    – Glen_b
    Commented Oct 8, 2017 at 0:56

2 Answers 2

Reset to default
7
$\begingroup$

If the only information you can glean from your strings is that they represent ranked lists (a < b < c etc), then I would suggest you replace the strings by their list of ranks (abddbabc -> [1, 2, 4, 4, 2, 1, 2, 3]) and use Spearman's correlation, or another rank correlation such as Kendall's tau.

Since you have a lot of repeated characters in your strings, take care about what you do with tied ranks.

Improve this answer
$\endgroup$
0
$\begingroup$

This thread discusses some possible dependence metrics for non-numeric objects like text strings ... Random "words" game

Specifically wrt your interest in identifying a Pearson-like metric for linear association between non-numeric text strings, the cosine similarity function is equivalent to the Pearson as a measure of linear dependence. Here's the wiki discussion of it ... https://en.wikipedia.org/wiki/Cosine_similarity

Personally, I don't agree with use of linear metrics, particularly wrt text mining. The nonlinear dependence metrics discussed in the thread above seem much more suitable to me.

Improve this answer
$\endgroup$
3
  • 6
    $\begingroup$ This answer doesn't seem to address the circumstances described in the question or the ensuing comments. Please note that although these are strings, they are not used or interpreted in the same way as most strings: they are a way of encoding sequences of numbers (that is, vectors). $\endgroup$
    – whuber ♦
    Commented Oct 7, 2017 at 16:20
  • $\begingroup$ @whuber One always has to be careful when reading your comments. In this instance you diplomatically note that my response "doesn't seem to address the circumstances" but that is not the same thing as calling out this response as "erroneous." Until the title to the OPs query is edited not to read, "Is there any way to compute Pearson's correlation between two strings?" as it does now, my view is that in suggesting use of the cosine similarity metric I more directly addressed the OPs query than anyone else on this thread. >> $\endgroup$
    – user78229
    Commented Oct 8, 2017 at 12:48
  • 1
    $\begingroup$ Moreover and again in my opinion, while converting text strings to ordinal numeric values is one way to approach the problem, it is not the only way. More importantly, this approach blurs the difference in magnitude between an 'A' and an 'F' to an order statistic of 1 and 2 within a string when, in fact, the distance between 'A' and 'F' is quite likely to be considerably greater than 1 unit across the full set of text strings. Given that, I respectfully submit that my response did, in fact, address the circumstances. $\endgroup$
    – user78229
    Commented Oct 8, 2017 at 12:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.