I am wondering what the potential pitfalls are of the following approach.
I want to check whether a neural measure is related to a questionnaire measure. I know that this is rarely the case (correlations are rarely significant and always small), and I know that I can collect a relatively underpowered sample (around 70 people) for the neural measure. However, I can collect as many as I like for the questionnaire measure.
Suppose I gave the questionnaire to many people, then went on to select a sub-sample for the neural measure: 30 at each tail-end of the distribution and 10 around the mean. Would this be problematic?
I believe it might artificially inflate the correlation if one exists, but my research question is whether there is a relation at all, and not how big it is. I also believe it wouldn't artificially create a correlation where none exists in the population. But I'm still unsure.
Does this sound like a reasonable approach? Are there better options? What are some problems with this approach?