I completed the ml-class offered by Prof Andrew Ng last fall I started trying one of the problems in Kaggle

When I fit a logistic Regression, the algorithm gave me poor results.

I saw that people do lot of data analysis before applying any machine learning algorithm

I completed a book in Statistics and want to learn about Data Analysis

Please help me identifying resources/courses that I can take to learn Data Analysis

  • $\begingroup$ Can you clarify how you understand machine learning & statistics to differ from data analysis? From your question it's difficult to figure out what you want to know. $\endgroup$ – gung - Reinstate Monica Feb 26 '12 at 18:28
  • $\begingroup$ okay to make it clear, when I see dataset, what analysis should I run first before applying any machine learning algorithms $\endgroup$ – learner Feb 26 '12 at 18:34
  • 3
    $\begingroup$ Your question is pretty broad. See if part of it is answered by the dozen or so threads that come up when you search on resources, sources, and recommendations. As far as "what analysis to run," that is so broad that perhaps all one can say is to read widely, study, experiment, consult with others, and give it time. $\endgroup$ – rolando2 Feb 26 '12 at 18:54
  • 2
    $\begingroup$ This question is not very clear. To reiterate what everyone else is saying: what do you mean by data analysis, and how does that differ from machine learning and statistical analysis? It sounds like you might mean preprocessing when you say that "people do a lot of data analysis before applying any machine learning algorithm". It seems like it would be extraordinarily difficult to complete a statistics textbook without learning anything about data analysis. If you want to learn about preprocessing data, personally I can't help you: in my experience it's usually been very problem specific. $\endgroup$ – Phillip Cloud Feb 26 '12 at 22:32
  • $\begingroup$ Please be more specific -- such discussions are better fit for chat. $\endgroup$ – user88 Feb 29 '12 at 8:38

Here a couple of thoughts:

  1. I think what you mean by data analysis is really exploratory data analysis. I can't think of any book better than Tukey's book of the same name. There are other books (Cleveland's "Visualizing Data" is an example), but Tukey is amazingly clear and full of insight.
  2. Another related area is to learn to clean your data. That's something you won't learn from Kaggle's data, because they are clean. Coming up with reasonably-sized (something that easily fits into your computer's memory) dirty data (eg. with missing data, conflicting items,,or duplicate records) is not easy. But if you could get your hands on such data, cleaning it would be really instructive. Knowing how to use relational databases (like MySQL) and writing queries in SQL can be helpful in the cleaning process (for instance, you can identify duplicate records).

I hope this helps

| cite | improve this answer | |
  • $\begingroup$ As using messy data: Just take world bank data for all countries by year. Then add UNO or whatever institutions Data to itt. Lots off fun: Different naming conventions for countries, missing data correlated to countries and lots of non normal distributed data. That was my master's thesis - have fun ;) $\endgroup$ – Christian Sauer Mar 17 '14 at 11:57
  • $\begingroup$ Might you share the title of Tukey's book? $\endgroup$ – Christian Sauer Mar 17 '14 at 11:57
  • $\begingroup$ Exploratory Data Analysis $\endgroup$ – user765195 May 24 '14 at 0:27

Not the answer you're looking for? Browse other questions tagged or ask your own question.