# In principal component analysis, is PC a linear combination of input variables or vice versa?

In every literature I've read each principal component is expressed as a linear combination of input variable. And the coefficient matrix is called factor loading. But why in John Hull's book (where he quoted Fyre 1997), an input variable is expressed as a linear combination of principal components? So he concluded that "When there is one unit of that factor, the 3-month rate increases by 0.21 basis points, the 6-month rate increases by 0.26 basis points, and so on". Is this just another convention or something? Thanks!

• PCA allows for going both ways. – Richard Hardy Mar 12 '18 at 20:35
• Or, to put it another way, the inverse of a linear transformation is also a linear transformation. – whuber Mar 12 '18 at 20:42
• Okay, I see. It makes sense. Since there's no correlation among PC's, does that mean the inverse of the linear transformation always exists and thus the two conventions are always equivalent? – Catiger3331 Mar 12 '18 at 20:48
• The "no correlation" is orthogonality of eigen vectors matrix. They're also orthonormal, so the inverse is a simple transpose. – Aksakal Mar 12 '18 at 20:53
• @Aksakal That's a good point, but to stave off a possible misconception it's worth noting that the eigenvectors have to be suitably normalized for your statement to be true--and that is just another way of stating that the transformation is a little more complicated than orthogonal. In particular, it shows why (and exactly how) the transformation may fail to be invertible. – whuber Mar 12 '18 at 21:52

Consider the purpose: why do in finance we extract the principal components from interest rate series? One reason is that we plug the factors to recover the rates, i.e. we take the modified output of PCA to get the new inputs. This may sound strange, so I'll elaborate.

One of the applications of the PCA applications is in risk management for the interest rate shock scenarios. Suppose that you have a portfolio of assets, that are sensitive to interest rates, and want to understand what is the interest rate risk of the book. One way of doing this is through scenario analysis.

If you had just one interest rate $r$ on the portfolio, you could look at the risk measure of the portfolio $V$ called the duration: $$D=-\frac 1 V \frac{\partial V}{\partial r}$$

Now, you could say that if the rates change by $\Delta r$, then the portfolio value would change by $\Delta V\approx-DV\Delta r$

The problem's that there's more than one interest rate, like in your exhibit we have rates from 3 months to 30 years. So what is $\Delta \bf r$ vector then? Of course, you could go with a so called parallell shock, i.e. set all coordinates of the $\Delta \bf r$ vector to an equal amount. This is quite popular, and still in use, especially in unsophisticated institutions and regulatory exercises a lot. However, we could try something more sophisticated.

One way to come up with this vector (shock scenario) is to use PCA. It happens so that if we run it on historical realizations of changes $\Delta r_{it}=r_{it}-r_{i,t-1}$ of the rate tenor $i$, the first principal component PC1 will account for up to 80% of the variance. Thus we can use this component to build the interest rate shock scenario, e.g. we could take one $\sigma$ shock up as follows: $$\Delta \bf r=\lambda\sigma_1 PC1$$ Here $\sigma_1$ - the standard deviation of the first principal component score, $\lambda$ - standard deviation explained by PC1 and $PC1$ - the loadings that you show from Hull's book. We could also use this to construct yield curve steepening/flattening shocks by using the slope component PC2. PCA allows to do this kind of interesting scenario construction.

So, to summarize, we first plugged the interest rates (inputs) into PCA and got the factors (outputs). Next, we modified the factors (outputs) and plugged them back into PCA and using coefficients (loadings) got the interest rates (originally, inputs) back. We went forward, then backward using the same PCA coefficient matrix!

• I just wonder if you need to divide the changes by its value as in $\Delta r_{it}=r_{it}-r_{i,t-1}$. If you put data of different tenor together (eg. 1 month and 30 years), because their absolute value differs a lot, you are not going to get a nearly constant factor loading for first PC as in the Hull's example. – Catiger3331 Mar 14 '18 at 15:23
• In PCA you can scale the volatility (e.g. by variance), it's a standard option in most stat packages. In practice the volatilities are not that different to actually do this. In my work the differences in PC from correlation or covariance regression are trivial. I don't have Hull's book with me, but think that the exhibit is actually on rates themselves, no scaling – Aksakal Mar 14 '18 at 17:39
• I checked the 5th ed of Hull's "Options, Futures..." book, Table 16.3, the factor loadings are for straight interest rates, no scaling was applied as I suspected. It's just not worth the hassle usually. – Aksakal Mar 14 '18 at 17:47
• Yes. Hull used interest rate daily differences. I don't know if I do wrong or not, but I don't get the nearly constant factor loading for PC1 as in Hull's example when I try to verify this with data from 2001 to 2018. The factor loadings are increasing with tenor. – Catiger3331 Mar 14 '18 at 19:28
• The factor loading will be changing day to day. You're not going to get constant loading most of the time. Whether it's a problem or not depends on your application. I personally do not bother about this in my applications – Aksakal Mar 14 '18 at 19:34