In principal component analysis, is PC a linear combination of input variables or vice versa? In every literature I've read each principal component is expressed as a linear combination of input variable. And the coefficient matrix is called factor loading. But why in John Hull's book (where he quoted Fyre 1997), an input variable is expressed as a linear combination of principal components? So he concluded that "When there is one unit of that factor, the 3-month rate increases by 0.21 basis points, the 6-month rate increases by 0.26 basis points, and so on". Is this just another convention or something? Thanks!

 A: Consider the purpose: why do in finance we extract the principal components from interest rate series? One reason is that we plug the factors to recover the rates, i.e. we take the modified output of PCA to get the new inputs. This may sound strange, so I'll elaborate.
One of the applications of the PCA applications is in risk management for the interest rate shock scenarios. Suppose that you have a portfolio of assets, that are sensitive to interest rates, and want to understand what is the interest rate risk of the book. One way of doing this is through scenario analysis. 
If you had just one interest rate $r$ on the portfolio, you could look at the risk measure of the portfolio $V$ called the duration: $$D=-\frac 1 V \frac{\partial V}{\partial r}$$
Now, you could say that if the rates change by $\Delta r$, then the portfolio value would change by $\Delta V\approx-DV\Delta r$
The problem's that there's more than one interest rate, like in your exhibit we have rates from 3 months to 30 years. So what is $\Delta \bf r$ vector then? Of course, you could go with a so called parallell shock, i.e. set all coordinates of the $\Delta \bf r$ vector to an equal amount. This is quite popular, and still in use, especially in unsophisticated institutions and regulatory exercises a lot. However, we could try something more sophisticated.
One way to come up with this vector (shock scenario) is to use PCA. It happens so that if we run it on historical realizations of changes $\Delta r_{it}=r_{it}-r_{i,t-1}$ of the rate tenor $i$, the first principal component PC1 will account for up to 80% of the variance. Thus we can use this component to build the interest rate shock scenario, e.g. we could take one $\sigma$ shock up as follows:
$$\Delta \bf r=\lambda\sigma_1 PC1$$
Here $\sigma_1$ - the standard deviation of the first principal component score, $\lambda$ - standard deviation explained by PC1 and $PC1$ - the loadings that you show from Hull's book. We could also use this to construct yield curve steepening/flattening shocks by using the slope component PC2. PCA allows to do this kind of interesting scenario construction.
So, to summarize, we first plugged the interest rates (inputs) into PCA and got the factors (outputs). Next, we modified the factors (outputs) and plugged them back into PCA and using coefficients (loadings) got the interest rates (originally, inputs) back. We went forward, then backward using the same PCA coefficient matrix!
