1
$\begingroup$

I am dealing with the question how to detect repetitive patterns in questionnaire answers. The idea is to filter out cases with clear patterns like

1,2,3,1,2,3,1,2,2,1,2,3,... or 4,9,4,9,4,9,4,8,4,9,... or 5,5,5,5,5,5,2,5,5,5,5,...

I am not sure which topic this is. Signal detection? Pattern recognition? Any hint would be great :-)

P.s. A quick method using R would be nice.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

The patterns you mentioned can be discovered with a an array shift.

Using python notations, for example:

[1,2,1,2,1,2,1,2,1,2,1,2]

Is the same if we cut 2 items from the left and paste them to the right.

Lets denote pattern length (2 in the example) as $l$

def similarity(arr,l):
    """Returns the percentage of the array that is matched with the pattern"""
    rotated=[arr[l:]+arr[:l]]
    identical_entries=len([1 for a,r in zip(arr,rotated) if a==r])
    return float(identical_entries)/len(arr)

The function above returns the percentage of items that follow a pattern of size $l$

Now, all we have left is the check several pattern sizes

for l in range(10):
   print ("For size {}, the similarity is {}".format(l,similarity(arr,l)))
Improve this answer
$\endgroup$
4
  • $\begingroup$ Hi uri, and thanks fo the idea. Indeed that could be an approach. The idea is not to need a given pattern as you suggest because then you have to test a lot of patterns giving a questionnaire of 54 questions with 5 choices. At the moment I am testing an approach to calculate entropy. $\endgroup$
    – Markus Graf
    Commented Sep 14, 2015 at 7:19
  • $\begingroup$ Note that this code doesn't test if given pattern repeats, it tests weather a pattern of a certain length repeats $\endgroup$
    – Uri Goren
    Commented Sep 14, 2015 at 8:08
  • $\begingroup$ But then you could also compare the maximum of unique answers with the maximum uniq answers of the pattern. Anyway I just realized that there is a bias towards the mean so both approaches are affected by this bias I guess. $\endgroup$
    – Markus Graf
    Commented Sep 14, 2015 at 8:42
  • $\begingroup$ And there is also the shannon-index that calculates the diversity $\endgroup$
    – Markus Graf
    Commented Sep 14, 2015 at 12:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.