Let's say I gather a ton of data in an experiment and then notice an interesting pattern unrelated to what I was testing.
So we know well enough that I can't suddenly form that pattern as the hypothesis and then use that data as evidence to test that hypothesis and get a p-value—I will need to run a new experiment.
So far so good? Okay, now consider the following approach:
Once you notice the pattern, don't tell anyone anything about it (not even subtle hints).
Just go up to one colleague who isn't likely to guess what you might have observed and ask "Is there any hypothesis you'd like to test on a dataset of this sort?" and see what they say:
If they suggested a different hypothesis, then your "freebie" is gone and you can't go ask someone else (otherwise you'll eventually find someone who suggests the same hypothesis). Tough luck, but that's how it works.
But if they happen to suggest the same hypothesis as yours the first time, then why not just run the test on the existing data you gathered? As long as you don't let them glean any information about the dataset from your question, it should be exactly the same as gathering new data from scratch, except that it saves all the costs, right?
Is this approach statistically valid? On the one hand it's probably making some of you cringe (me too) that we're depending on the likelihood of a human to guess the "correct" hypothesis (we're essentially turning things around and testing the experimenter!), but on the other hand from a mathematical standpoint I don't see why it shouldn't work...