I have a spectrum data (wavelength(x) versus absorption(y)) for 25 unique samples that is almost exactly to the problem presented in this thesis: https://brage.bibsys.no/xmlui//bitstream/handle/11250/2371385/12296_FULLTEXT.pdf?sequence=1&isAllowed=y.

I have about 10 samples that are one kind of biological sample and 15 that are another. For each sample there are 6 highly similar but different replicates (for a total of 25x6 datasets).

See attached image of one.enter image description here I can handle the test and validation sets, however I don't know how to handle the replicates.

My thoughts are to:

  1. Average the Y values to produce a single dataset,
  2. Randomly pick one replicate for each sample and throw the others away,
  3. Treat all replicates as unique individual observations (essentially 125 of them), or
  4. Something else.

I suspect that; #1 is invalid, #2 will end up losing a lot of data, and #3 is statistically improper.

What is the correct solution?


First, depending on the source of the data, you may want to do some normalization (standardization, integral normalization, etc.) to make the spectra comparable, specially across biological replicates. (Also possibly background/baseline correction.)

I would start with your approach (3) and plot the results in a low dimensional space (MDS or PCA) just to see if the technical replicates are not too dispersed, and cluster together. As I understand, in both (1) and (2) you end up with the same number of samples (=25), and the standard procedure is to average the technical replicates. You may find this article describing biological and technical replicates to be of help.

But again, I believe that the signal preprocessing may have a strong effect. In some cases (e.g. hyperspectral imaging) it is valid to use all "replicates" (e.g. pixels) from each sample, although supervised methods are commonly employed.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.