4
$\begingroup$

I've found a nice presentation describing PLS1 and PLS2 algorithms (pages 16-19). It's pretty clear but there is a thing confusing me.

For PLS1. Let's look at the algorithm. The first steps are

  • $w = X'y$ (which maximizes $\operatorname{cov}(Xw,y)$)
  • $w = w / ||w||$
  • $t = Xw$
  • $p = X't / t't$

On the one hand one can say $T=XW$ and $WW'=I$, but on the other hand $T=XP$ and $PP'=I$ (because we are searching for a decomposition $X=TP'$ where $PP'=I$, see page 14).

So, my question is, aren't $P$ and $W$ the same matrix? And if so, why does the algorithm need to calculate $p$ as $X't / t't$?

Why not do it this way:

  • $p = X'y$
  • $p = p / ||p||$
  • $t = Xp$

UPDATE After reading provided comments, answers, and links (thanks to @amoeba and @theGD), I get that a strict answer to my question is "No, they are not.". I almost understood why. Actually, I lost my hope to fully understand NIPALS algorthm principle. So, I decided to ask it in a differernt way: What is mathematicial task PLS NIPALS tries to solve?

For example, there is NIPALS for PCA as well. And I don't understand it fully too. But I know that it's just a computational method for solving a mathimaticial task (for one iteration): $$ {\mathbf {t}}^{T}{\mathbf {t}} = {\mathbf {w}}^{T}{\mathbf {X}}^{T}{\mathbf {Xw}} \rightarrow \max, \textrm{ given that } \Vert {\mathbf {w}}\Vert =1 $$ So, what is analogous mathematical task for PLS?

$\endgroup$
  • 2
    $\begingroup$ No, they are not the same. You can easily verify it using your own favourite programming language: generate random $X$ and $y$ (they can be very small), then carry out first steps of this algorithm: compute vector $w$, then vector $t$, then vector $p$. You will see that $w\ne p$. $\endgroup$ – amoeba says Reinstate Monica Mar 23 '17 at 12:57
1
$\begingroup$

The 14th page is not about PLS, it is about PCA since it refers to decomposition of $X$ alone by carrying out singular value decomposition which is an efficient way to obtain eigenvalues($D$) and eigenvectors($V$) of $\operatorname{cov}(X)$). So in PCA case, the orthogonality of $P$ is correct.

In PLS, however, the only property of $X$ loadings is each vector in $P$ matrix has unit length ($||p_i||$ = 1) whereas $W$ is orthogonal. In fact, the addition of $W$ is to ensure the orthagonality of $X$ scores($T$) and it is one of the main differences between PLS NIPALS and PCA NIPALS. Thus, as said in comments, $P$ and $W$ are not the same matrices.

As a small note I prefer SIMPLS algorithm which is faster and it provides a single vector for regression and makes interpretation a lot easier unlike NIPALS with multiple vectors each corresponding to deflated $X$ which is, in my opinion, quite counter-intuitive and hard to interpret.

The properties of PLS factors obtained by NIPALS algorithm can be found in this article: Geladi, Paul, and Bruce R. Kowalski. "Partial least-squares regression: a tutorial." Analytica chimica acta 185 (1986): 1-17.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.