Constructing a Bayesian network from the begining

Question

I am trying to construct a bayesian network which detects fraud.I have got a huge data set, with elements such as country, top spend merchant just to name a couple. The first part of my problem is to use a test set of data which is about 11,000 rows in excel and use this to firstly use Bayes formula to see the $P(\text{Fraud}|\text{each variable})$.

After I have done this I have to perform this on the rest of the data and see if it is a good algorithm. This is the first part that I am confused with. How do I do this?

I know $P(\text{Fraud}|X_1)=P(X_1|\text{Fraud})\frac{P(\text{Fraud})}{P(X_1)}$ and I dont understand why I have been told to do this on a test set of data.

Answer 1

Computing P(Fraud | x_i) for each attribute will only tell you which attributes directly provide information about fraud. This can be useful in variable selection (e.g. if you want to use a naive bayes classifier to classify transactions as fraudulent or not), but to learn the dependence structure of the set of all available variables, you'll have to do more work.

If you're goal is to construct a complete Bayesian network, then the quality measures will need to consider the likelihood of the data given the network structure and parameters (how well does one configuration of variables explain the data as compared to another configuration). This is a rather involved procedure for which I would direct you to this tutorial for more information.

If you are familiar with Matlab, there is a fantastic toolbox that implements structure learning. Kevin Murphy, the primary author of the toolbox, also has a bunch of tutorials and example code on his website that may be useful.