Why would exploratory data analysis be important to undertake before null-hypothesis tests?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Perhaps if there is no precise hypothesis to test, the research should always be considered exploratory. $\endgroup$– FlaskJan 10, 2014 at 1:54
-
$\begingroup$ Usually you wouldn't do both on the same data (or at least not on the same subset). If you already have a specific hypothesis with a kind of data you understand (e.g. you've seen other samples of it), you may not need EDA at all. If you have no hypotheses or don't have any idea what assumptions might reasonable, it's sensible to look at data to generate hypotheses/assumptions ... but then you don't test the hypotheses you generate on the data that generated them (at least not unless you can adjust for that). One solution is sample splitting (EDA on part of the data, testing on another). $\endgroup$– Glen_bJan 10, 2014 at 10:36
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
It is often necessary to know a little about the system being explored before sensible hypotheses come to mind and it is very useful to know about the variation and noise in an assay prior to designing an experiment. Exploratory experiments and analyses are good for that. Don't be too quick to decide that a dataset is definitive.
Of course, you should know that hypotheses that are suggested by the data in exploratory analyses will have a high chance of giving you a spurious 'significant' result if you test them using the same data, so ideally the exploratory analyses lead to the design and running of new experiments to specifically test hypotheses.
answered Jan 10, 2014 at 0:47
-
$\begingroup$ How can you avoid this issue? Should you split your data into an EDA and hypothesis dataset? $\endgroup$– ConnorApr 4 at 17:19
-
1$\begingroup$ In my usual context (biomedical laboratory experimentation) the process usually allows for separate gathering of data for preliminary exploratory analysis and for the formal testing of hypotheses. (I'm not saying that all experimenters work that way...) Yes, I would expect that a random slitting of a large dataset into preliminary and hypothesis testing groups would be equivalent. $\endgroup$ Apr 4 at 20:47
$\begingroup$
$\endgroup$
There really aren't rules on which comes first: data-driven (hypothesis-generating) analyses then hypothesis-driven analyses, or hypothesis-driven followed by data-driven.
If you knowingly want to test hypotheses and then do knowledge discovery, then you can answer questions you have, and then learn from (data-driven analyses) a part of the data that is novel (never been studied before) in order to generate hypotheses.
Otherwise, if you needed to run exploratory first to generate hypotheses, then if nothing is found -- no patterns, no clusters, no correlations -- essentially noise, then you wouldn't be able to test any hypothesis since they wouldn't exist.