3
$\begingroup$

I work with 4 sequence datasets that I want to compare. The datasets range from roughly 150 to 2000 sequences. I've been able to use TraMineR on datasets up to 1000 sequences long, but when I try to run the dataset with 2000 sequences, even our University's supercomputer cannot finish in under 36h. Hence, I'm considering sampling by id variable. My dataset looks like this:

id event   time
1  edit    1
1  edit    2
2  comment 3
3  comment 4
4  close   5

Now, I want to take a random sample of, say, 2, by id numbers. Let's say the random sample yields id 1 and 4, then my sample would look like this:

id event   time
1  edit    1
1  edit    2
4  close   5

In other words, I'm sampling 2 id numbers, but I'm actually getting 3 cases.

Now, on to my question: I only need to do this sampling for 1 of my 4 datasets, and I'm doing it for computational reasons. What are the considerations that needs to go into the sampling design? For example:

  • Should I draw an equally large sample from every dataset (by % or N?) even if I don't have any problems computing on the smaller datasets?
  • How large should the sample be in order for me to be able to claim generalizability?
  • Would it be OK to simply sample as many cases as I can handle from the largest dataset, then run the other datasets without sampling (since they are small enough to compute on as they are), and then make comparisons across all 3 datasets?
$\endgroup$
  • 1
    $\begingroup$ Which command takes hours? If this is seqefsub, try setting maxK=3 (4, or 5). It limits the length of the subsequences we are looking for (we only search for subsequences of length 3, 4 or 5). Most of the time, longuer subsequences are not of the main interest. $\endgroup$ – Matthias Studer Jan 20 '14 at 18:55
  • $\begingroup$ It's seqecreate and seqformat. The data is 3'552 sequences containing 117'727 activities. And that's just 6 months worth of data - I have multiple years of data - so I think I'll need to sample regardless of if I can speed up individual TraMineR commands upsomewhat $\endgroup$ – histelheim Jan 20 '14 at 19:58
3
$\begingroup$

The only point you may want to care is representativeness of your data.

Assuming you want to respect the size of your four datasets in the analysis, you can use weights. Most of the tools proposed by TraMineR are taking account of provided weights when applicable. (Weights are provided with the weights argument of seqdef for state sequences, and are assigned to event sequences with the seqeweight function.)

In your case, it is OK to simply sample the largest dataset. If you sample say 20% of the data, you would then apply a weight of 5 to each sampled sequence from the largest dataset and could then mix those sequences with unit weighted sequences from the three other datasets without biasing outcomes.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.