It's a class project (however, I want to expand it to make it bigger than just a class project for my own sake) and the professor gives us freedom to choose whatever we want as long as it includes some dataset (deal with them using R) and analysis of it. Since I'm starting to become interested in stock market, algorithmic trading, etc. I was wondering whether I can find some such projects here. My professor now doesn't have some specific project though.
closed as too broad by Nick Cox, John, Nick Stauner, Scortchi♦ Feb 23 '17 at 16:08
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
Ok...I'll bite. First, there are a multitude of resources out there containing free market data at various levels of detail and composition. For instance:
Yahoo! Finance provides easily downloadable past 12 months price information for all US traded stocks and, if you know the underlying CUSIP, you can pull down information about bonds too
Robert Shiller's website has a wealth of historic information going back as far as the mid-19th c. http://www.econ.yale.edu/~shiller/data.htm
Kenneth French's website has portfolio level information http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html
CRSP - Center for Research in Security Prices has another boatload of data http://www.crsp.com/ although I don't know if it's free
iVelocity.com has information about stock options
Quandl.com has literally millions of financial time series indexes from an enormous range of sources
It might be cool to mash up a bunch of stock, macroeconomic indexes, etc., to see what kind of relationships emerge.
Next, if you want to impress your professor, don't use ANOVA for the analysis. It has linear, Gaussian assumptions wrt the residuals and it's well known that the tails of financial data are way more extreme than Gaussian. So, try something neat like Hyndman's boosting additive quantile regression as described in this paper, "Forecasting Uncertainty in Electricity Smart Meter." (http://robjhyndman.com/papers/smart-meter-quantiles.pdf). Forget the "electricity" part, it's just a novel methodology that can be applied to financial information. Once the quantiles have been fit, you can assume any underlying probabilistic, extreme value distribution that you like.
Didier Sornette's book "Extreme Financial Risks" is an excellent overview of extreme value theory as applied to financial markets. In addition, his ETHZ website has many more amazingly creative papers well worth browsing (http://www.er.ethz.ch/financial-crisis-observatory.html).
So, hopefully this helps. At a minimum, you have pointers to some useful resources for data. The extreme value theory suggestion may not be that helpful, depending on your comfort level with its highly technical demands.
/** Additional Data Sources **/
The St Louis Fed has a large archive of macroeconomic indicators https://research.stlouisfed.org/econ/mccracken/fred-databases/
The US Financial Diaries has hhold level information about consumption patterns http://www.usfinancialdiaries.org/
The Billion Prices Initiative is described as an, "Academic initiative that uses prices collected from hundreds of online retailers around the world on a daily basis to conduct economic research. This page shows our most recent research leveraging high-frequency price data, as well as the US daily inflation index." http://bpp.mit.edu/
There is a problem with your plan. You cannot use ANOVA on stock market data. The statistical distribution involved in stock market data has been in dispute since 1963. That ended recently with someone doing a first principles derivation of the returns that would exist under a variety of circumstances. With the narrow exception of firms being merged out of existence for cash, you cannot use ANOVA.
The distribution of returns, for equity securities that are going concerns, must be some form of transformation of the Cauchy distribution which is then truncated at -100%. You can show that the CDF of the Cauchy distribution is the limit of one of the integrals when the problem is converted to polar coordinates. In any circumstance, the solution is either a Cauchy distribution, or a deformation of the Cauchy distribution.
The difficulty is that the Cauchy distribution lacks a mean, and therefore also lacks a variance. At first this is a strange idea, because you can add numbers up and divide the sample size and therefore you have a sample average. You, of course, have sample averages, but they mean nothing because there is no population average. Because returns are truncated your center of location is the mode and there is no standardized term in use for the scale parameter because truncation disrupts simple descriptors like the inter-quartile range.
It better to think of the variance as a feature of a distribution than as something that must exist, like noses are a feature of most vertebrates. If you see something without a nose, its probably not a vertebrate. This also renders most algorithmic trading methods invalid.
The field will have years of clean-up to do. While you should master ANOVA as it is a very useful tool, do not use it with stocks or anything else that grows at an exponential rate, such as cancer. You do want to master this tool because of its intrinsic value, nonetheless.