10
$\begingroup$

I am reading an example given in the book, Machine Learning for Hackers. I will first elaborate on the example and then talk about my question.

Example:

Takes a dataset for 10 years of 25 share prices. Runs PCA on the 25 share prices. Compares the principal component with the Dow Jones Index. Observes very strong similarity between PC and DJI!

From what I understand, the example is more like a toy to help newbies like me understand how effective a tool PCA is!

However, reading from another source, I see that share prices are non-stationary and running PCA on share prices is absurd. The sources from where I read totally ridicule the idea of calculating covariance and PCA for share prices.

Questions:

  1. How did the example work so well? The PCA of share prices and DJI were very close to one another. And the data is real data from 2002-2011 share prices.

  2. Can someone point me to some nice resource for reading up on stationary / non-stationary data? I am a programmer. I have a good math background. But I haven't done serious math for 3 years. I have started reading again about stuff like random walks, etc.

|cite|improve this question
$\endgroup$
10
$\begingroup$

This piece serves to partly answer the original question and some of the questions raised in comments to @JonEgil's answer.

Financial (logarithmic) returns* are approximately $i.i.d.$ (although there is often some conditional heteroskedasticity) -- while prices are approximately random walks. Under the assumption of $i.i.d.$ observations, principal component analysis would directly generalize from sample to population (i.e. the sample principal components would be estimating the population principal components), but this might not hold under non-$i.i.d.$ observations -- see this thread. This is why it makes sense to run PCA on (logarithmic) returns rather than prices.

Ruey S. Tsay has argued for running PCA on residuals from econometric models of financial time series, since residuals are normally assumed to be $i.i.d.$ I think that this idea might be included some place in his "Multivariate Time Series Analysis with R and Financial Applications" textbook (he explained the idea to me in person, so I am not sure where it is written).

* Logarithmic return on price $P_t$ is defined as $r:=\text{log}(P_t)-\text{log}(P_{t-1})=\text{log}\frac{P_t}{P_{t-1}}$. Logarithmic returns are used for convenience in place of percentage returns $r':=\frac{P_t-P_{t-1}}{P_{t-1}}$. The convenient feature of logarithmic returns is that you may sum up $h$ individual logarithmic returns to get the total logarithmic return over $h$ periods, while this does not hold for percentage returns. For relatively small percentage returns (which is common in finance), logarithmic returns approximately equal percentage returns as the logarithm has approximately unit slope around one.

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ +1, this is interesting. Can you expand a bit on what actually is a "return"? My knowledge of economics is zero; I googled and found that if price is given by $f(t_i)$ time series, then returns are defined as $\log\frac{f(t_{i+1})}{f(t_i)}$. Is that correct? If so, then why the logarithm? I would understand your argument about the relation between iid returns and random walk prices if returns were defined as differences. Apart from that, DJ is the average price, so I still don't understand why PC1 of returns should be a better match than PC1 of prices, even given your considerations about iid. $\endgroup$ – amoeba says Reinstate Monica Dec 7 '15 at 20:56
  • 1
    $\begingroup$ @amoeba, I added a quick explanation and have to leave now. I hope I did not make too many mistakes there. I will be back tomorrow if there are any further issues. $\endgroup$ – Richard Hardy Dec 7 '15 at 21:06
  • 1
    $\begingroup$ Thanks. I see now that returns (logarithmic returns) are essentially a derivative (first difference) of the logarithm of prices. So if the claim is that returns are iid and log prices are random walks, then it makes sense. However, I am still surprised by the Dow Jones example and would appreciate any further clarifications. $\endgroup$ – amoeba says Reinstate Monica Dec 7 '15 at 21:16
6
$\begingroup$

I run these types of analysis professionally and can confirm that they indeed are useful. But please make sure you analyse returns not prices. This is also highlighted by the critique in Slender Means: 

To perform PCA, your data have to have a meaningful covariance matrix 
(or correlation matrix, but the conditions are equivalent). They analyze 
stock prices, which are non-stationary time series variables.

A typical usecase in our analysis is to quantify systemic risk in the market place. The more co-movement in the market, the less of a diversification you really have in your portfolio. This can, for example, be quantified by the amount of variance described by the first principal component. Which is identical to the value of the first eigenvalue.

For financial data, one typically examines a moving window over time. Some form of decay factor that downweights older observations is useful. For daily data, anything from 20-60 days, for weekly data maybe 1-2 years, all depending on your needs.

Note that for global financial markets, with tens- or hundreds of thousands of asset prices changing continuously, one typicall can't run a 100K vs 100K covariance matrix. Instead, typical usecase is to run the analysis per country, per sector or other more meaningful groups. Alternatively break down the return by a set of underlying factors (value, size, quality, credit ....) and do the PCA / Covariance analysis on these.

Some nice articles include Attilio Meucci's discussion on effective number of bets: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1358533

, and also Ledoit and Wolf's Honey I shrunk the sample covariance matrix http://www.math.umn.edu/~bemis/MFM/2014/spring/References/lw_shrinkage.pdf

For a financially oriented introduction to stationarity, why not start with Investopedia. It's not rigorous, but conveys the main ideas.

Good luck!

EDIT: Here is a 3-stock example showing Apple, Google and Dow Jones with daily returns through 2015. The upper triangle shows correlation of return, the lower triangle show correlation of prices.

Upper triangle return correlation, lower triangle price correlation

As can be seen, Apple has a higher price-correlation with Dow (bottom left 0.76) than return correlation (top right 0.66). What can we learn from that? Not much. Google has a negative price correlation with both Apple (-0.28) and Dow (-0.27). Again, not much to learn from that. However, the return correlations tell us that Apple and Google both have a fairly high correlation with the Dow (0.66 and 0.53 respectively). That tell us something about the co-movement (price-change) of assets in a portfolio. That is useful information.

The main point is that although price correlation can be just as easily computed, it is not interesting. Why? Because the price of a stock is not interesting in itself. The price change, however, is very interesting.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Can you please expand more on the main part of the question which is about the difference between using prices vs. returns? I understand that when using prices, correlation matrix will be influenced by the non-stationarities; e.g. if all prices linearly grow, then all correlations will be strongly positive. First, why is it bad? In particular given that Dow Jones is essentially an average price and it will grow too (as PC1 will). Second, how is using returns supposed to help? AFAIK "returns" are logged ratios of neighbouring points; why is it meaningful and how is it related to Dow Jones? $\endgroup$ – amoeba says Reinstate Monica Dec 7 '15 at 17:27
  • $\begingroup$ thanks for your informative reply. But it doesn't answer my question. I want to understand why analysis on price is working very well for the data set in the book? And amoeba has raised a lot of valid questions. $\endgroup$ – claudius Dec 7 '15 at 17:28
  • 1
    $\begingroup$ @claudius: The fact that PCA on prices gives something similar to Dow Jones which is the average price is not surprising at all. I'm rather wondering why PCA on returns produces a better fit. Perhaps Jon will be able to clarify. $\endgroup$ – amoeba says Reinstate Monica Dec 7 '15 at 17:29
  • 1
    $\begingroup$ I have not looked at the actual code run in ML for Hackers, but whenever someone say they analyse prices, 99 times of of a 100 what they actually analyse is log-returns. For example, today the Dow is down 162 points, while Apple is down 0.88 dollars. Not only are the numbers vastly different, they even are on a different scale, index points vs money. But in pct terms 0.91% and 0.75% are comparable and the numbers you want to work with. For some analysis, one can de-trend the data by subtracting the mean. In short term financial timeseries this is often ignored, assuming no trend. $\endgroup$ – Jon Egil Dec 7 '15 at 17:45
  • 1
    $\begingroup$ @amoeba, To (partly) answer the questions raised in comments, returns are approximately i.i.d. while prices are approximately random walks. Principal components have their nice properties under the assumption of i.i.d. observations. This is why it makes sense to run PCA on returns rather than prices. Ruey S. Tsay has argued for running PCA on residuals from econometric models of financial time series, since residuals are normally assumed to be i.i.d. I think that might be included some place in his "Multivariate Time Series Analysis with R and Financial Applications" textbook. $\endgroup$ – Richard Hardy Dec 7 '15 at 20:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.