# PLS Regression On 3D Matrix

When doing PLS regression, you normally have a 2D matrix where each row is a sample of known class identity (in the case of a training set) where the 2D matrix is the X, and the class information is the Y. However, my dataset is different. I have a bunch of 2D spectra combined into a 3D matrix. Each entire 2D matrix has a class identity (not each row). In the case of this higher dimensional (3D instead of 2D) dataset, how can I do PLS regression? The first step besides mean centering is to multiply the 1D Y vector by a 2D matrix via matrix multiplication, but instead I have a 3D matrix. Is it possible to multiply a 1D and 3D matrix together?

## 2 Answers

I would give vectorization a chance.

Simply put each spectra right after the other for each sample then do the PLS.

Let me give an example: Assuming you have 100 spectra for each sample where each spectra has 300 wavenumbers. Putting each near another would lead to 100*300=3000 variables in total for each sample. From there you may try doing regular PLS.

I am no expert in hyperspectra. This is, however,what I usually do and works most of the time for me on images where there are 2D pixel matrix as well as 3 RGB color information in each pixel. I think yours is a similar case.

First of all, the methods you are looking for are called multiway methods in chemometrics, and there is a whole lot of literature on this. Geladi: Analysis of multi-way (multi-mode) data, Chemometrics and Intelligent Laboratory Systems, 7, (1989) 11-30. may be a good starting point to learn about these methods.

As @theGD says, you can unfold the N-way array and then perform "normal" PLS. This is e.g. suggested by Wold, Geladi, Esbensen, and Öhman: Multi-way principal components-and PLS-analysis, Journal of Chemometrics, 1, 1, 41–56 (1987)..

However, there are other algorithms availabe, see e.g.

Answer to @theGD's comment/question: unfortunately I don't have first-hand experience with multi-way as so far - our experiments usually have nested factors (dimensions/directions) rather than the crossed experimental layout to which you can apply multi-way analysis. So take the following with a grain of salt - these are just my very general thoughts on the subject.

In general, I'd expect any better performance to come from additional constraints you can include into your N-way model, or putting it the other way round, from not modeling the interactions between your factors. This implies that you can gain performance only if it is sensible to specify the model without interactions.

So assuming e.g. fluorescence absorption-emission spectra of a system with 2 species, i.e. 2 spectral + 1 sample dimension.
Say, a data cube of $n_{samples} \times n_{wl.ex} \times n_{wl.em}$.

Unfolding this into $n_{samples} \times (n_{wl.ex} \cdot n_{wl.em})$ and decomposing with 2 latent variables (chemical rank = 2) will yield a spectra² matrix that still contains structure. I.e. the matrix covering the unfolded 2 spectral dimensions can itself still be decomposed into absorption and emission spectra of the 2 species.

So while the unfolding version describes the original data cube by two matrices of size $n_{samples} \times n_{species}$ and $n_{species} \times (n_{wl.ex} \cdot n_{wl.em})$, an 3-way model describing the system by three matrices of size $n_{samples} \times n_{species}$, $n_{species} \times n_{wl.ex}$ and $n_{species} \times n_{wl.em}$ would be sufficient. That is, far fewer ($n_{wl.ex} + n_{wl.em}$ vs. $n_{wl.ex} \cdot n_{wl.em}$) parameters are estimated on the spectral dimensions by model 2 compared to model 1.

Assuming 100 samples, 2 species and 300 wavelengths for both excitation and emission, we have 2 ⋅ (100 + 300 + 300) = 1400 parameters in total for the 3-way model and 2 ⋅ (100 + 300 ⋅ 300) = 180200 parameters for the unfolded model which are estimated using a total of 100 ⋅ 300 ⋅ 300 = 9⋅10⁶ elements in the data cube (raw measurements). That is about 50 raw measurements per estimated parameter in the unfolded model vs. $\approx$ 6400 raw measurements per estimated parameter in the 3-way model.

You can also describe the unfolded model as containing both factors (excitation and emission) as well as all their interactions - whereas the 3-way model does not include the interactions.

• Unfolding was the word I have been trying to find! Do multiway methods perform better overally compared to unfolding? – theGD Jan 31 '17 at 11:50
• @theGD: please see the updated answer: as always in data analysis, it depends... – cbeleites unhappy with SX Jan 31 '17 at 13:43