Bear with me as I try to word this question well (I'm a mathematical modeler, but not a statistics guru).

We want to assess the response patterns of a binary timeseries. The data are from people answering questions for information they have not previously seen. The scenario is that answers are either correct or incorrect, and they have to get an answer correct twice in a row before it is "retired." Since people have not previously seen the information, the "perfect" pattern is 0-1-1, meaning the person answered incorrectly, then was presented the correct answer/information, then retained that for the next two attempts.

I'm trying to assess people's "trend" and am seeking a pointer to how this might be possible. It's not enough to collect responses, since two people having 3 correct responses and 3 incorrect responses might reflect very different "trends." To wit:

  • Person A: 0-0-1-0-1-1
  • Person B: 1-0-1-0-1-0

In the first case, the person experiences a trend toward understanding. In the second, the person is alternating with no trend toward understanding. Ideally, we'd come up with some number normalized to assess some distance from the perfect 0-1-1 pattern. For instance, 1-0-1-1 would be more distant than 0-0-1-1, but closer than 0-0-0-1-1. Even a distance from 0, 1, or some other arbitrary point would be good if it was internally consistent.

My first thought would be to look for something in signal analysis– assess the periodicity of the signal of responses. However, I was wondering if the statistics community has some manner to assess a binary sequence like this. Of course, I'm not looking for an easy answer (since there might not be one) but pointers in the right direction would be helpful.

Edit to address @gung's questions:

I don't expect that the timeseries will be more than 10 to 20 attempts max. Mean number of attempts per question seems to be about 2.8 for well written questions. The series will be different for each user's response to each question, with thousands of users answering generally on the order of 10 questions each (i.e somewhere between 1 and 100, depending on how long they participate).

I'm looking to rank individual users per question and per group of questions, but also try to rank questions by groups of users (i.e. this question was poor because many users flipped back and forth between 0 and 1).

  • $\begingroup$ How long are these series going to be, & how many of them will you have? Will they vary in length for different people? $\endgroup$ Commented Oct 18, 2012 at 23:26


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.