3
$\begingroup$

I'm kind of new doing data mining, so sorry if my question is not very clear.

I'm working in a project that is aiming to do data mining over the interactions of the students with a e-learning platform. So, I'm trying to generate decision trees with some data that I collected (number of times that student use a resource, number of activities done, number quizzes taken, etc). The trees that I'm getting are interesting for me and according with the cross-validation tests, it's kind of accurate.

So my doubt is the following, I generated the tree with the data collected over two months and there were 44 students in the course. Is this enough data to trust in the tree? I have only the 44 instances that gather the two months of interactions...

Thanks in advanced for your help.

$\endgroup$
2
$\begingroup$

{I'm guessing that for each person and each variable you have a single value covering the 2 months--i.e., you don't have repeated measures.} 44 sounds like an awfully small number. Is it a random sample of a larger population? The answer to that, for one thing, would affect your confidence in the findings.

I won't say it's impossible to achieve useful, replicable results in this situation, but it seems unlikely. Data mining via decision trees is an opportunistic process and requires crossvalidation, probably even more than other modelling procedures do.

I'd also like to know, if you're getting statistically significant differences that form the basis for the branchings, what criterion are you using for significance?

$\endgroup$
5
  • $\begingroup$ Hi Rolando, thanks for answer. Your guess is right. For each person I have single value covering the two months. The 44 people were the students that enrolled to this unit. It wasn't random sample, basically it was all the population that I had access. I'm planning to repete the experiment for future students... but right now I would like to grab some partial conclusion of the behavior observed... The last question I don't understand it... do you think that I should use other approach instead of decision trees?? thanks a lot for your help $\endgroup$ – Rafael Apr 17 '11 at 16:40
  • $\begingroup$ Sounds as if you could try to make the case that this year's students constitute a random sample of the population of all students who might enroll. That way you'd be justified in generalizing from sample to population...I was asking what was your basis for getting the tree to split into branches. I'm familiar with programs that give you the option to form a split if certain criteria are met....How did you crossvalidate, though, if you only had 44 cases? $\endgroup$ – rolando2 Apr 17 '11 at 20:51
  • $\begingroup$ Hi I used 10-fold stratified cross-validation. The software that I'm using have this option to train and test the model. Are you familiarized with this kind of cross-validation? $\endgroup$ – Rafael Apr 18 '11 at 14:54
  • 1
    $\begingroup$ I may well be missing something, but if you're splitting a sample of 44 cases into 10 subsamples, doesn't that leave only 4 or 5 per subsample? If so, it's unfortunately a situation where the software makes something possible that doesn't have any practical worth. Perhaps the procedure you're using is more like bootstrapping or jackknifing, where you select most of the 44 each time, sampling with replacement? $\endgroup$ – rolando2 Apr 18 '11 at 19:31
  • $\begingroup$ Mm is true... I'm only leaving 4 to 5 instances per subsample with this approach... it doesn't makes much sense. I'm going to check the bootstrapping and the jackkinifing to see what I can find. Thanks a lot for you answer! $\endgroup$ – Rafael Apr 18 '11 at 20:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.