10
$\begingroup$

In my free time, I am working on a small web based system which collects crash reports (but not other, non-crashing bug reports) that are sent from Delphi Windows applications.

For troubleshooting, users would love to have a data-mining feature to find relationships between hardware or operating system versions and the specific bug and/or crash.

As an example how this should work:

  • for every crash there is a report in the database, which has a fingerprint / hash code of the stack trace (call stack) at the moment of the crash to identify duplicates
  • the algorithm checks if all duplicates of a bug report also have some other common attributes, for example a missing service pack of the operating system
  • the analysis result lists all properties which the bug reports have in common

Let's assume these automatic bug reports contain all key information like the names of all processes which are currently running, file names, version information of loaded DLLs, etc.

How can I find correlations between repeated crashes and the environment? Are there specific algorithms or statistical methods that would help?

$\endgroup$
7
  • 2
    $\begingroup$ Correlation finding is a hefty task. Have you seen this tool at work ? It might help you find the proper way of organizing your data. $\endgroup$
    – Carlo Kuip
    Commented Oct 20, 2011 at 17:39
  • $\begingroup$ So you want to do the correlation computation in Delphi, too? Then take a look at this list of Delphi math functions: it covers a lot and also gives explanations and links to the libraries. $\endgroup$
    – DaveBall
    Commented Oct 20, 2011 at 21:47
  • $\begingroup$ @DaveBall Thank you for the interesting link, however while the bug reports come from Delphi, the web service actually is implemented in Java - maybe I can use the algorithms as a starting point, or in an 'offline' stats tool. $\endgroup$
    – mjn
    Commented Oct 21, 2011 at 4:42
  • 1
    $\begingroup$ You might want to look into "discrete hazard models." This seems like a case where you want to regress failure events (crashes) on fixed characteristics (environmental variables) and the time since the last crash. If you have enough observations (crash reports), you will be able to add interactions between the environmental variables to the model. These will then point you in the direction of system conflicts. Unfortunately, I don't know how easy it is to implement such a model in Java. $\endgroup$
    – jmtroos
    Commented Apr 5, 2012 at 14:46
  • 1
    $\begingroup$ The kinds of analyses you could conceivably employ depend on the size of the problem i.e. How many distinct kinds of crashes are there? How many possible attributes are there? $\endgroup$
    – Sameer
    Commented Jun 25, 2012 at 4:34

1 Answer 1

1
$\begingroup$

Could you sample your user's [non-crashed] machines for the same info as you get in a crash report? Because then you could use logistic regression to model those attributes (and interactions) to the probability of getting a crash.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.