3
$\begingroup$

I'm solving the spectroscopy problem. Based on reflectivity values for wavelengths from the spectrum, I build a regression to find a target for the sample.

I have 30 samples. For each sample I take measurements of its spectrum 3 times. In total, I get 90 elements in the dataset, but in fact, 2/3 of the elements repeat the existing spectrum with a small error. Target for each sample is almost not repeated, distributed from 0 to 100.

The question is:

1)How to split multiple measurements of the same sample between folds? (should I use GroupKFold?)

If, for example, we make LOO cross validation, then we have two measurements of the same spectrum in the training set and its third measurement is in the test set. As a result, we have a very small error. 2)But wouldn't it be a "leak" ?

3)Or should all three measurements of the same sample be in the same fold only?

Thanks.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

The best way would be (3), i.e. by respecting the dependence between data samples. Otherwise, if their co-dependence is strong, you might end up with optimistic results.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Do I understand you correctly? Do you propose, in my case, to divide 90 measurements into 30 folds so that one fold has only 3 measurements of one sample, and so do cross validation? $\endgroup$
    – Mishin V.
    Commented Aug 5, 2020 at 13:04
  • 2
    $\begingroup$ You don't need to do 30 folds but assure that data for each sample is always in the same fold. $\endgroup$
    – gunes
    Commented Aug 5, 2020 at 14:11
  • 2
    $\begingroup$ Yes, you need to split samples, not spectra. To implement this on sample level, GroupKFold is your friend. $\endgroup$
    – cbeleites
    Commented Aug 6, 2020 at 18:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.