Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 9 down vote favorite

In my free time, I am working on a small web based system which collects crash reports (but not other, non-crashing bug reports) that are sent from Delphi Windows applications.

For troubleshooting, users would love to have a data-mining feature to find relationships between hardware or operating system versions and the specific bug and/or crash.

As an example how this should work:

  • for every crash there is a report in the database, which has a fingerprint / hash code of the stack trace (call stack) at the moment of the crash to identify duplicates
  • the algorithm checks if all duplicates of a bug report also have some other common attributes, for example a missing service pack of the operating system
  • the analysis result lists all properties which the bug reports have in common

Let's assume these automatic bug reports contain all key information like the names of all processes which are currently running, file names, version information of loaded DLLs, etc.

How can I find correlations between repeated crashes and the environment? Are there specific algorithms or statistical methods that would help?

share|improve this question
2  
Correlation finding is a hefty task. Have you seen this tool at work ? It might help you find the proper way of organizing your data. – Carlo Kuip Oct 20 '11 at 17:39
So you want to do the correlation computation in Delphi, too? Then take a look at this list of Delphi math functions: it covers a lot and also gives explanations and links to the libraries. – DaveBall Oct 20 '11 at 21:47
@DaveBall Thank you for the interesting link, however while the bug reports come from Delphi, the web service actually is implemented in Java - maybe I can use the algorithms as a starting point, or in an 'offline' stats tool. – mjn Oct 21 '11 at 4:42
1  
You might want to look into "discrete hazard models." This seems like a case where you want to regress failure events (crashes) on fixed characteristics (environmental variables) and the time since the last crash. If you have enough observations (crash reports), you will be able to add interactions between the environmental variables to the model. These will then point you in the direction of system conflicts. Unfortunately, I don't know how easy it is to implement such a model in Java. – jmtroos Apr 5 '12 at 14:46
1  
The kinds of analyses you could conceivably employ depend on the size of the problem i.e. How many distinct kinds of crashes are there? How many possible attributes are there? – Sameer Jun 25 '12 at 4:34
show 1 more comment

migrated from programmers.stackexchange.com Feb 5 '12 at 20:17

1 Answer

up vote 1 down vote

Could you sample your user's [non-crashed] machines for the same info as you get in a crash report? Because then you could use logistic regression to model those attributes (and interactions) to the probability of getting a crash.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.