PLS identify only peaks not troughs, and ignore certain region

Question

I recorded a few Raman spectra for varying concentration of a substance. I processed the data in R and these are my steps:

1. Remove baseline using baseline.corr with lambda=1e3 and p=0.01
2. Run PLSR on (concentration ~ spectra)
3. For the first few components (3 in this example), pick the wavenumbers that have the highest scores in the loadings. (Having a high score = contributing significantly to the variance in data between concentrations)

My goal is to try to identify peaks of my chemical automatically using PLS regression.

I've plot the graphs out for everyone to visualize:

Description of the graph:

• The blue line represents the original data
• The black line represents the spectral data fed into PLSR after preprocessing
• The red sections represents the wavenumbers with the highest loading scores from the top few principal components.

As you can see, this works fairly well.

I would like to improve my algorithm to do the following:

1. Ignore region < 500. This is because my CCD is at those ranges and the data received is unreliable. I don't want PLSR to take into account that area. (An obvious answer would be to zero that out, but I would prefer a solution that involves having some sort of parameter to tune PLSR to pay less attention to those areas)
2. For the wavenumbers identified in step 3, I would only want peaks and not troughs. In raman spectroscopy we are more concerned with the presence of signal than the absence for identification of chemicals.

What can I do to achieve these two goals?

Answer 1

Ignore region

External knowledge such as the fact that your instrumental set-up cannot measure reliably outside certain spectral ranges, or you're working on a substrate/in a solvent that renders certain regions unsusable, or that for the type of sample you have, no bands at all can appear in certain regions is very valuable knowledge.

PLS is fairly good at recognizing such regions itself provided you supply training data that exemplifies this knowledge, but IMHO you'd usually be better off to feed in this knowledge manually and focus your experiments on variation that will occur in practice rather than on artificial situations.

I'd say you're in a no-free-lunch situation here: you can ask your PLS model to do this, but you'll have to pay for it.

Having PLS paying less attention to region

You can scale that region to downweight it:

yarn.pls <- plsr(density ~ NIR, 6, data = yarn)
yarn.dw <- yarn
yarn.dw$NIR[, 100:150]  <- yarn.dw$NIR[, 100:150] / 10 
yarn.dw.pls <- plsr(density ~ NIR, 6, data = yarn.dw)
plot (coef (yarn.dw.pls), type = "l", col = 2)
lines (coef (yarn.pls), type = "l", col = 1)
rect (100, -10, 150, 10, border = NA, col = "#00000020")
abline (h = 0, lty = 2)

Get a model that uses only bands

IMHO this request is not sensible: Identity of the analyte is established both by present Raman bands as well as by the absence of bands characteristic for other structural elements. Think of how you conclude qualitatively that you got an alkane sample: you find C-H and C-C (stretching and deformation) vibrations, and no vibrations due to other functional groups.
So in order to reduce cross sensitivities you need to establish the whole spectral pattern.

PLS is an inverse calibration (regression) technique, so it is meant to be as robust as possible wrt. to the presence of other, even unknown, substances. In that sense, your request is not in line with what PLS is supposed to do.

Get a model that uses only a few wavenumbers

PLS is a regularization that is meant to produce latent spectra which cover basically the whole spectral range. However, there are other regularization techniques which basically provide variable selection, e.g. the LASSO.

Large weights/coefficients/loadings

For the first few components (3 in this example), pick the wavenumbers that have the highest scores in the loadings. (Having a high score = contributing significantly to the variance in data between concentrations)

Yes, but unless your data is variance scaled (which is usually not a good idea for spectroscopic data) you need to take into account also the intensities you have at the respective wavenumber. You may get a huge weight/coefficient for a small but important band while an equally important strong band will get far smaller weights and coefficients.

---

Update: A few thoughts about identifying substances in a possible mixture

• Please read up on the advantages and disadvantages of ordinary vs. inverse calibration: your choice in that respect should depend on your actual application, in particular whether you can reasonably assume that you can collect reference spectra of all substances that can possibly be in the mixture or not.

• I'm not even sure whether regression/calibration is the right technique. If you aim at qualitative identification of substance (classes) rather than at concentration measurements, you may be better off with one-class classification.

• To establish identity of a substance, I insist that you need presence as well as absence of bands/peaks, at least if there is no external information that helps narrowing down the choices.
  Here's an example. Say, you measure some carboxylic acid and your training spectra contain some alkane. Looking for presence of alkane bands only, you'd get a hit on alkane (meaning, really, that vibrations of C-C and C-H bonds are detected)?
  (That SERS due to the local and directed enhancement makes things somewhat more complicated doesn't help here, neither...)

  Note that this is a very basic caution to the interpretation of mixture vibrational spectra.

  In the framework of one-class classification, you'd be much closer to the usual interpretation of spectra: you'd detect presence of C-C, C-H, and COOH/COO⁻ for the carboxylic acid, all of which are correct. And in this framework absent bands are not needed (other than saying which part of the spectrum is not yet explained).

Answer 2

This serves as a reply to @cbeleites's post

Thank you for your answer. I've chosen to write another response because my comments for your answer is rather lengthy and it's just easier to write it here.

First I would like to clear up some misinformation in my question above: The graph above is actually the signal from SERS enhancement. The SERS signal are the peaks that you can see under the 500 range. In the multiple Raman scans that I've collected, my SERS peaks fluctuates significantly across multiple readings. I am fully aware that SERS is a highly localized phenomena, so it is very difficult to achieve uniformity across different readings (aka the SERS uncertainty principle). Hence, I want to be able to tell PLS to pay less attention to those regions. You solution of "down-weighting" the signal actually makes a lot of sense in this case.

Second of all, I am interested to identify only peaks ("bands" in your vocabulary) because I am trying to build a database of spectra of known chemicals for a research project. My goal is to do the following: given a sample of mixed chemical, will I be able to tell what is present based on the database of spectra from different chemicals I've collected? I agree with your example that the absence of certain signal is an important part of identifying a chemical. However, consider this scenario: Chemical A has peaks at 300 and 500 and Chemical B has peaks at 400. If I have a mixture of containing A and B, I should only be checking the presence of peaks 300 and 500 for A and 400 for B against the signal of my sample. The absence of 400 in A's signal does not serve a purpose here because the presence of signal at 400 does not imply the lack of A in the mixture.

Regarding your comments on data scaling, I did not realize this behavior of PLS. If variance scaling my data is not recommended, what about scaling each wavenumber independently? The result of this operation will allow PLS to properly treat each wavenumber's difference across rows without being interfered by other wavenumbers with higher intensities.

What will be the best chemometric models for my pursuit?