Data Preparation – Historical yearly data rolled up or kept as separate observations I have a data set regarding audit and tax takes which I want to do some analysis (possibly clustering and predicting who to audit in the coming year).
The data ranges from 2009-2013. I have (up to) five observations per company (for each year) which contain data financial return data pertaining to each year. Every few years a company gets randomly audited and there is a recorded amount of money yielded or fined from the company.
My question is how do I lay the data out when performing this analysis. Should I roll each company up in to one row with all of the financial return data or should keep each year separate. Obviously any year that the company isn't audited the yield will be 0. However if they were not audited in 2010 but were audited in 2011 and yielded £10,000, how should I deal with the 2010 observation (assuming we have left them as separate observations)? Should I have a variable holding any previous or future yielded amount even if it didn't happen that year?
Thanks.
 A: To address your first question (aggregate vs keep separate): is there any reason you couldn't do both?  Why not just add an aggregate column next to the individual years' results and see which performs better?
I am going to change your second question a bit (having some experience in the same field): do you (the company) really want to predict whether or not companies will file a [false/erroneous/fraudulent etc] or do you want to prevent companies from doing so?  If you do want to prevent this, you should probably perform a cost benefit analysis. Given that some companies file [false/erroneous/fraudulent etc] reports: which is more expensive, sending out auditors or accepting incorrect reports?  On the other hand, if you simply want to predict: build a model, create your predictions, and wait for those fake reports to roll in!  Better yet- make it an office pool!  (ok, now I'm being facetious)
To address your underlying premise (predictions via audit data):  in my experience, trying to create a prediction from an audit data set is not a productive activity.  Is the point to use financial returns to predict the result of the audit?  I would think the two are largely uncorrelated.  There are possibly some independent variables that could be used to predict the results of the audit- e.g. how stringent their internal controls and self-inspection are- but these are difficult to quantify.  I realize you might not have had much choice in the assignment, so just a word of caution not to expect a high degree of correlation.
