7
$\begingroup$

Are there some papers published which illustrate EDA used to tackle substantial data problems? I am particularly looking for actual (current) data examples, where plots have been made and statistics computed that reveal things in the data that we would not have been able to detect otherwise, or with models. Here are a couple of examples of what I am interested in finding. Both of these examples show things that were discovered in data by making plots. I'd also be interested in discoveries made by rough calculations, like Tukey used to do, eg like median polish. Not from fitting models, where lots of assumptions are required.

This is an old example, from a data set on tipping in restaurants, see introduction of ggobi book for the full example,

enter image description here

with the observation that "many diners round tips to the nearest $1 and 50c value". The peaks in the histogram with the small bandwidth occur at regular intervals, too much to be due to chance. Hand et al found similar behavior when mining a large credit card data set, when customers were purchasing petrol in the UK. He followed up the discover by setting up a model that had multiple components, one with the rounding behavior and another following a more regular distribution.

See Hyndsight blog for a recently released statistics on unemployment. This is the critical picture:

enter image description here

with the observation, "that there is something different about Aug this year." The most plausible explanation is a change in the way the unemployment is being collected.

Improve this question
$\endgroup$
7
  • $\begingroup$ This is a great question. I'm not sure if it is going to get the attention it deserves. Remind me in a couple of days & I can put a bounty on it to drive more views. $\endgroup$
    – gung - Reinstate Monica
    Oct 11, 2014 at 16:34
  • $\begingroup$ BTW, PubMed's database can be queried automatically to look for papers regarding "exploratory data analysis". The URL should have the format: http: //www.ncbi.nlm.nih.gov/pubmed/?term="[text-to-be-found]" I have prepared a link to get the first 200 results related to EDA. Google Scholar can be also of help if you configure an alert. $\endgroup$
    – Ray G. Butler
    Oct 13, 2014 at 11:36
  • 1
    $\begingroup$ I like this question, too--but I just cannot justify keeping it open as it is currently formulated. It asks for an undifferentiated, open-ended list of answers and provides no effective criteria to evaluate their suitability or appropriateness. If you can think of a way to overcome these obstacles, I'm sure many people would like to reopen this question. $\endgroup$
    – whuber ♦
    Oct 13, 2014 at 16:29
  • 1
    $\begingroup$ I'd really like to keep this question open. I wouldn't consider the answers given so far to be opinion-based. Perhaps, slightly misunderstanding of what EDA is, but this is good to know too. One of the bigger problems is that some of the examples given are not easy to get access too - would be better to have examples that are available openly on the web. So, I am going to add some examples with my question, hopefully to clarify the question. $\endgroup$
    – Dianne Cook
    Oct 15, 2014 at 1:10
  • 3
    $\begingroup$ I understand the position that this thread is primarily opinion based. How will an accepted answer be selected, eg? However, I do think there are (very limited) cases where such threads are acceptable & even useful for CV. I am voting to re-open primarily based on my opinion that this thread is one of those cases. $\endgroup$
    – gung - Reinstate Monica
    Oct 15, 2014 at 2:00

2 Answers 2

Reset to default
4
$\begingroup$

One example I enjoy (and is a simple illustration) is the work by Michael Maltz on analyzing the uniform crime reports that police agencies supply to the FBI. See:

Maltz, M. D. (2010). Look before you analyze: Visualizing data in criminal justice. In Piquero, A. . and Weisburd, D., editors, Handbook of Quantitative Criminology, chapter 3, pages 25-52. Springer New York, New York, NY.

For some background, the FBI does not have standardized ways to report missing or incomplete reports (they collect data monthly, so an agency could report for some months but not the entire year). So the uncritical would observe zeroes or very low numbers for a particular jurisdiction and not presume missing data, e.g. see the numbers for Florida in Parker & Pruitt (2000). So there is quite a bit of precedent in the criminology literature of modelling this data without discovering such errors.

Here is a good example from blogs discussing published papers:

  • Uri Simonsohn on the Data Colada blog and Felix Schönbrodt on a failed replication in pyschology and how ceiling effects of the instrument are not an issue. Here are the images of the original and replication ECDF's from the Data Colada blog:

Original Replication

There are also some good examples on this site. I thought I had a good example here but a few others that I really enjoyed are:

I realize these aren't published, but I think are illustrative nonetheless. I'm sure you could cull up more on this site as well.

Improve this answer
$\endgroup$
1
0
$\begingroup$

Our neuroscientist and colleage Dr. Trejo had a successful experience with exploratory data analysis applied to adult neurogenesis in his work "Involvement of specific adult hippocampal neurogenic subpopulations on behavior acquisition and persistence abilities" (under peer-review so details cannot be provided still).

I would suggest you to contact Dr. Trejo and chat with him. He's truly cooperative so I'm sure he will explain his case with more detail.

The issues they were facing in his lab were two:

  1. Their data only showed "trends" about their original hypothesis on the relationship between neural structure and learning-memory processes.

  2. They spent weeks manually looking for relevant correlations with classical statistical packages.

(Automated) Exploratory data analysis helped them to found some hints on were could be the key variables involving the relationship. Of course, a further confirmatory work was done then.

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ This isn't really an answer anyone can use. If details can't be provided, then posting an answer should be delayed until publication. It also seems to be promotional. $\endgroup$
    – Nick Cox
    Oct 13, 2014 at 11:36
  • $\begingroup$ I understand that Dianne is not looking for the details about the publication itself but for the experience of using EDA in publishing a paper. Anyway, the author contact details are provided so that a further discussion can follow. $\endgroup$
    – Ray G. Butler
    Oct 13, 2014 at 11:38
  • 2
    $\begingroup$ Answers should be self-contained. Giving references is fine; suggesting that you contact a person is quite a different thing. But the validity of your answer is for moderators to decide. Meanwhile please review advice on self-promotion at stats.stackexchange.com/help/behavior $\endgroup$
    – Nick Cox
    Oct 13, 2014 at 11:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.