I am trying to construct a bayesian network which detects fraud. I have got a huge data set, with elements such as country, top spend merchant just to name a couple. The first part of my problem is to use a test set of data which is about 11,000 rows in excel and use this to firstly use Bayes formula to see the $P(\text{Fraud}|\text{each variable})$.

After I have done this I have to perform this on the rest of the data and see if it is a good algorithm. This is the first part that I am confused with. How do I do this?

I know $P(\text{Fraud}|X_1)=P(X_1|\text{Fraud})\frac{P(\text{Fraud})}{P(X_1)}$ and I dont understand why I have been told to do this on a test set of data.

  • $\begingroup$ @kjetilbhalvorsen, please don't approve the spam edits w/o proper discussion on meta.CV. It may well be that the tag should be changed. I'm indifferent to that. However, this should be discussed on meta, & the tag would be made a synonym, rather than have a person arbitrarily decide that a tag encoding nearly 100 threads be changed all at once. $\endgroup$ – gung - Reinstate Monica May 1 '16 at 20:34

Computing P(Fraud | x_i) for each attribute will only tell you which attributes directly provide information about fraud. This can be useful in variable selection (e.g. if you want to use a naive bayes classifier to classify transactions as fraudulent or not), but to learn the dependence structure of the set of all available variables, you'll have to do more work.

If you're goal is to construct a complete Bayesian network, then the quality measures will need to consider the likelihood of the data given the network structure and parameters (how well does one configuration of variables explain the data as compared to another configuration). This is a rather involved procedure for which I would direct you to this tutorial for more information.

If you are familiar with Matlab, there is a fantastic toolbox that implements structure learning. Kevin Murphy, the primary author of the toolbox, also has a bunch of tutorials and example code on his website that may be useful.

  • $\begingroup$ Thank you! I have read through the first tutorial and was wondering, since I have physical data, do I need to model the data as distributions to determine the prior and likelihood or can I use my data, e.g. p(fraud)=no fraudulent transactions/no transactions? $\endgroup$ – maths May 17 '12 at 10:52
  • $\begingroup$ You can use maximum likelihood estimates (this results in simply counting # of transactions that include fraud = yes, normalized by the total number of samples to estimate p(fraud = yes/no)). $\endgroup$ – Nick May 17 '12 at 20:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.