How to prove that calibration is not data dredging Sometimes multiple tests are conducted of the same hypothesis, using the same model each time but with some model parameter changed.  As results for similar values of the parameter are similar, this can be viewed as a calibration of the model rather than data dredging folly.  Here's an example:

The graph above shows 21 different tests.  Applying Bonferroni correction to these tests would probably render all findings insignificant (i.e. instead of a 1% confidence level require a 1/21 = 0.048% confidence level to attain significance).  But, as adjacent test results in parameter space are strongly correlated, intuitively we haven't conducted 21 independent "tests" here - maybe closer to 5?  Is there a way to quantify the effective number of independent tests in this manner?
** UPDATE **
Thanks for two good answers below.  The graph above is illustrative - in this case I had sound physical reasons to believe correlation would have a smooth peak somewhere between 100 and 1500m, and on testing indeed it did, so I'm not very worried about it (if anything it confirms the mechanism).  And it also works on a different dataset altogether. :)
Let's suppose that the test had turned out differently though; that there was no peak where I expected it but a very good peak elsewhere.  Suppose the unexpected peak invalidated my model, though it could be explained by another model I'd thought about before (albeit didn't happen to be testing this time) - let's call it model X.  And suppose I had no alternative data for an independent test.
This is surely a situation that crops up in practice and to my knowledge there is no good answer to it.
Eupraxis' answer would then imply that I had been data dredging because of the issues with my model, and cbeleites answer would imply that I had to collect independent data to validate.
Is there in this situation, some way I can re-use the same data to validate model X?  Using the Bonferroni correction (or another such technique) as a formal way to "convert" a calibration exercise into multiple - but not independent - tests, and hence arrive at an honest measure of significance that won't give a false negative?
 A: Calibration and "data dregding" are not really differentiated by statistical tests, but by motives. If you are simply looking for patterns in an undirected manner, then you are data dredging. However, if you have a well formulated model that is known to capture the general features of a phenomenon, save for the specific values of some parameters, then what you are doing is calibrating. The difference is how well formluated your model is. If you are using the data to determine the form of your model, then you are essentially data dredging or pattern-finding (less negative connotation). This is exploratory data analysis.
An example of calibration from my own field of environmental engineering is when we want to relate the cloudiness (turbidity) of a facility's effluent to the total mass of solids in the effluent. The general form of the relationship is Total Solids = a*Turbidity. However, the coefficient a is variable, so we use field data to calibrate it. Once a is determined, is generally stays pretty constant as long as the facility's process is not changed. So, that is a concrete example. 
It all depends on whether or not you know the form of the model for the underlying process under study. That's how I've differentiated it in the past.
Expansion after OP's edits
I agree that if you found something unexpected and then happend to notice that another model fit, then you would be effectively data dredging. The problem with trying to justify your new model as statistically significant, post hoc, is that its hard to quantify the probability that you would have another model that happened to fit and that you would have noticed it. It's sort of like the situation when two friends randomly meet in a city and ask what are the odds of that...this is not an exact analogy by any means, but the point is that when you are dealing with complex data and many data points and you don't have a clear testing plan at taht start, you are entering a situation where your risk of spurious identification is high, but the exact "Type I" risk you are taking on is no longer quantifiable. At this point, the most you can really say is that an alternative model is suggestive, but you really need to get a validation dataset to be any more confident (i.e., I wouldn't go to publication based on such finding, unless I could indepenedently verify). If you feel that it is right, given your professional judgement, then by all means use it but know that you cannot use statsitics to back up your model at this point.
If you must get some kind of quantification, perhaps there is a way to simulate your data under the model you think is correct and see how often the alternative model would provide a better fit and vice versa. I'm not familiar with your problem, but that is one "gut check" that might be helpful.
A: 
the point is to pick a value of the calibration parameter that gives optimum model correlation

In that case, the question is about model optimization, which is related to, but usually not considered the same, as data dredging.
Fortunately, there are "recipes" that you can follow to avoid the problems associated with the dredging. 


*

*The big problem with both model selection/optimization and data dredging is that people forget to validate their final model with independent test data.
The problem is not primarily that lots of models are tested. IMHO you can try around as much as you like, as long as the final model comes with an independent test.
Incidentally, the difference between independent test results and "internal"/optimization results gives you an indication of how much overfitting you have.

*Thus, in oder to avoid the data dredging/optimization problems, you need to do two levels of testing. The inner test data is used for model selection/optimization, and once that is finished, you do the outer testing with data that did in no way contribute to the model. This includes that 


*

*splitting occurs at the uppermost level of the data hierarchy and 

*the set-aside test data does not contribute to data-driven pre-processing (centering, scaling, PCA projection, ...)
Keywords to look for would be nested/double validation. 


*A situation that occurs frequently in this context is that aggressive optimization overfits, and you end up with many models that seem perfect. In that situation the optimization/selection will not know which model to choose as they all seem similarly perfect. If then you observe a huge drop in performance estimate with the independent (outer) error estimation, you may be better off not optimizing at all but instead using your expert knowledge about the problem to fix the parameter in question. I write this here as your network radius in m probably has a "physical" interpretation, and you may know what order of magnitude would be sensible.
You should not be afraid of using this expert knowledge to fix the parameter and thus avoid all problems of data dredging! 

*In any case, I'd point out that the choice of the radius parameter is a "benign" problem, as you observe a rather broad plateau from 500 - 1000 m. Within this interval, nothing happens, which also means that you do not need to worry whether 605 m is better than 607 m. This is an important information, because for future data you may fix the radius, and argue that with that you avoid all possible data dredging/optimization problems and that this is sensible because you found this wide plateau in the previous study.
Note however, that looking for the maximum now and finding the plateau, you cannot any longer argue that you fixed the parameter beforehand! As you did this, you need to do the independent validation now.
Note that in calibration with models with low complexity and many independent data points (classical calibration as in @Eupraxis1981's answer), usually lack of fit is responsible for the error. That is, if at all, the model is underfit, but for sure not overfit. In that situation you may get away with resubstitution error (i.e. use the goodness of fit with your training data as approximation for generalization error).
