0
$\begingroup$

For a data set with 3 replicates of 9 different treatments (27 samples) what would be the best way to run the cross validation of PLSR models? There are not enough samples to have separate training and test sets.

My initial thought was to just do k-fold cross validation with 5 randomly generated segments. However after reading some of the information from Camo in the Unscrambler help menu they suggest that all segments should "contain unique information". Therefore should I use systematic segmentation and have 9 segments each with all replicates of one treatment?

Your opinions would be appreciated.

$\endgroup$
1

1 Answer 1

1
$\begingroup$

First of all, this question really cannot be answered in a "do this" or "avoid that" fashion without thorough knowledge about your application and data. A better question would be "When to use [splitting strategy]?".

But here's one very general strategy you can always apply: cross validate in all ways considered (there are several possibilities of systematic segmenting, e.g. venetian blinds vs. consecutive blocks and often also different sensible possibilities of stratifying random assignment). If the results are equivalent, you don't need to bother further. If you do see differences, it is time to dig in and find out what they mean.

Other than that, you need to consider what the differences between the splitting methods mean in detail for your application and whether that particular aspect of information is important.
E.g.: the blocked varieties (you can cut into consecutive blocks wrt. different variates: time of measurement, target variate, etc) allow you to check extrapolation just outside the calibration range (outer blocks of target variate) or an onset of instrument drift (outer blocks of measurement time) which sometimes is very useful information. Venetian blinds or random split results could serve in order to get a baseline for interpreting those results. Venetian blinds (again: according to which variate?) are basically the limit of stratification wrt. that variate.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.