I'm still a beginner at data mining. I'm working on finding the association rules from hypothesis X to conclusion Y. To this end, I've conducted a survey with questions that go something like this:

Q1: Do you have any relatives with ABC property? (Ans: yes or no)
Q2: Do you have an interest in XYZ field? (Ans: yes or no)
Q3: Which institute are you from? (Ans: Option 1, 2, 3, 4 or 5)

and so on.. So there are lots of "parameters" or "features" or "dimensions" to my data.

My data is formatted similar to this: http://www.hakank.org/weka/weather.arff, and I'll be using WEKA.

However I'm still currently in the data pre-processing stage. Removing duplicate entries and dealing with missing values is no issue. What I'm worried about is removing outliers.

Firstly, how can I represent this type of record data in such a way that similarity measures like Euclidean or Minkowski (or perhaps any!) distance can even be applied to it?

And secondly, what's the most reasonable similarity measure to use in this type of case? I've looked at the Mahalanobi distance and it seems useless for my project because I have no "ideal" set of features against which I could compare other sections of data. Is it usual to even need to detect outliers before finding association rules? Or are outliers usually detected after the rules have been learned?

I've been thinking about this for a while but can't seem to reach a sensible conclusion. Could any of the more experienced data miners help please?

  • $\begingroup$ @Miroslav Sabo, could you possibly help? $\endgroup$
    – user961627
    Commented Oct 31, 2013 at 11:44


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.