0
$\begingroup$

I want to apply a partially least square regression on spectroscopy data to model a chemical content of my probe. So, every wavelength of the spectrum serves as one variable in the model. Doing some research, I found several scenarios how to preprocess the data: Most just use the spectrums as they are and do no preprocessing at all. Others mention that it is important to do a column normalization (e.g. mean centering for every wavelength across the observations) before modeling. I also read that even the target variable should be normalized. Since normalization is important for dimension reduction, (what is one part of PLSR) it kind of makes sense for me to normalize the data. However, most examples I found people are not doing any normalizations.

What would be the best procedure?

Thanks for any help!

$\endgroup$

1 Answer 1

0
$\begingroup$

Most PLS tutorials found online consider the variables to be independent and have different scales (e.g. distance and temperature). Such cases require column normalization (over each variable). In spectroscopy, row normalization (over each spectrum) is more common because the intensity values are not independent and are in the same scale already.

Column normalization of spectra increases the importance of the wavenumbers that have low intensity and decreases the importance of the wavenumbers that have higher intensity, which leads to increased noise contribution.

If your input data are spectra, then the recommended way would be to avoid scaling data for every wavelength. I did not find any published research that shows it, but I know that from experience, and it is also stated that way for Raman spectra at https://www.nature.com/articles/s41596-021-00620-3 in the "Spectral truncation and normalization" sub-section.

As to the scaling/whitening of the target variable, it might be useful if you have multiple target variables with significantly different value ranges. However, some PLS algorithms are insensitive to the scaling of the targets.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.