Is publishing work based on post hoc analysis problematic? Nowadays, in the scientific domain, increasingly, people in the top position argue against doing post-hoc analysis and they advise against collecting a bunch of data and just make up a story after data collection and report the significant findings. In psychology, which is my field, recently some journals accept pre-registration of research proposals and those journals allow those studies to be published regardless of the results (see http://www.sciencemag.org/careers/2015/12/register-your-study-new-publication-option)
Yes, I understand people's conceptualisation of how science is done is that:


*

*I have some competing (maybe mutually exclusive) hypotheses

*I design a study (maybe an experiment) and collect data

*I look at the data and see if it supports my hypotheses

*I publish my findings


However, is there really a problem of some people's "post hoc" way of doing research? If there is a hidden pattern in reality and researchers couldn't really have a grasp of it at the moment, what's wrong with doing a exploratory study and collect a bunch of data and examine their relationships in a post hoc way? And why is making up a story to cover up a study a problem?
 A: Making up a story would be dangerous but, to your general point, it depends. It obviously complicates any statistical methods you use but in many cases it would be wasteful to not contribute new credible information. I would expect your finding to be treated with greater skepticism but few things are black and white.
A: Obviously  post-hoc analysis and explanatory studies provide useful pieces to understand and discuss phenomena : There is nothing wrong with them and with publishing them. The moment where such designs become a bad practice is when they are used for conclusions that only a properly dedicated study could support. Care must be taken when interpreting post-hoc analysis, for the data analyst as well as for the person that will interpret the reported result.  
Actually in most cases, post hoc explorations of the data fully immerse us in "The garden of forking paths" where it is almost impossible to conduct proper hypothesis testing (at least without increasing drastically type II error) as a large number of choices are then conditioned by the data (e.g. which variables to exclude, how to group data, what relationship to test?).  Indeed, in large data set, you can look at the data in a certain way (here I mean with the good grouping, thresholds, fitting function ..) so that a striking pattern appears. A pattern in data does not mean a pattern in the phenomum and only  a proper replication can justify a conclusion (as far as a hypothesis test can justify a conclusion...). 
A: There's a formula called "Bayes' Theorem" that says that if you start out assigning probability P1 to a hypothesis H, and you see evidence E, then you should adjust your probability to:
P2 = P1*(probability of seeing E given that H is true)/(probability of seeing E). 
So if something would normally be very unlikely to see, but probable if the hypothesis is true, then seeing it should significantly increase your confidence in the hypothesis. However, if something is likely to be seen regardless of whether the hypothesis is true, then it should not increase your confidence.
Statistical analysis can tell you what the probability of a particular study arriving at a result given the null hypothesis is, but that is not the same as the probability of it being seen. Unfortunately, the first number is erroneously treated as if it were the second. 
This difference is the basis of the Monty Hall paradox: if you pick Door A and are shown that Door B has a goat, then the evidence "Door B has a goat" is equally likely regardless of whether Door A has a car, so it shouldn't cause you to switch. However, "I saw that Door B has a goat" is less likely if Door A has a car, because in that case Monty Hall has only a 50% chance of showing you Door B. Therefore, while knowing that Door B has a goat shouldn't cause you to switch, knowing that you know that Door B has a goat should.
That is, if Monty Hall always shows you Door B, regardless of whether it has a goat, then seeing it's a goat shouldn't make you switch. But if Monty Hall never shows you Door B when it has a car, always shows you Door B when Door C has a goat, and randomly chooses between Door B and Door C when they both have a goat, then seeing that Door B has a goat should make you switch.
Similarly, if someone shows you the result of a study, and you can confidently say "I would have seen this statistic regardless of what it had been", then you can take the naive probability calculations at face value. But if you see a statistic, and you realize that this statistic probably wouldn't have been mentioned if it weren't so impressive, then now you have to adjust for that bias.
So if you have a rigorous, predetermined procedure by which you become aware of results, and the probability of seeing a result is not dependent on the results of that study, then you don't have to worry about this distinction between "Probability of E" and "Probability of knowing E". But once the probabilities diverge, now you have an extra parameter that you have to estimate, and you likely will have only a vague idea of what this parameter should be, and it will be very tempting to simply ignore this issue.
