0
$\begingroup$

I have a question about performing either 10-fold or leave one out cross validations with biological replicates.

In total I have 50 samples, each of which has four biological replicates. I am interesting in seeing how the samples are classified as either one class (0) or another (1).

My issue is how to correctly deal with the replicates: for example, would it be correct to partition the data such that one or two biological replicates for a sample are in the training set and supplemented with all other samples while the remaining replicates are in the test set?

Any help would be greatly appreciated!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

would it be correct to partition the data such that one or two biological replicates for a sample are in the training set and supplemented with all other samples while the remaining replicates are in the test set?

Typically: no

As a rule of thumb, the splitting of the data should always ensure complete independence of training and test data and therefore needs to be done at the highest level of the data hierarchy.

There may be special cases where you are interested in in-sample prediction (i.e. the model will never be used for new unknown samples), where in-sample testing would be appropriate, but I've not yet met a valid reason for doing so.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for your response. Just to make sure I understand this, if I am not interested in predicting unknown samples then it would be appropriate to partition the samples as mentioned in the original question? Wanted to add one more point: I came across this source which stated that "validation involved training sets based on six replicates on which the remaining two replicates have been projected." Is this approach the same as described in the question? $\endgroup$
    – tolo9397
    Commented May 18, 2016 at 8:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.