Analyzing data while collecting the data Part of our samlpe has been collected (Phase 1, $n=12$ subjects with measurements pre and post treatments). The remaining sample (Phase 2 with another 8-10 subjects) will be collected in a few months.
Both phase 1 and phase 2 are supposed to be identical in terms of study design and treatment. The only reason for the second phase is that we were not able to obtain the desired number of subjects for the first phase. The final analysis of the treatment effect is suppose to include all subjects from phase 1 and phase 2. 
While waiting for the rest of the data, is it ok to analyze the first part of the data? I feel hesitant to analyze the first part for the following reasons:


*

*The results of the interim analysis might affect the second phase. Depending on the results changes might be possible to the study design and/or treatment. However, an interim analysis (as in clinical trials) was never planned. A further concern is that the results might affect how/which patients are being allowed to participate in the second part. I doubt this would happen on purpose, but might be possible subconsciously and thus falsify the results.

*The results between the interim analysis and the final analysis might change. I might get a significant treatment effect for the phase 1 analysis which gets everybody excited. When analyzing the whole data the effect might vanish given the small sample. Now it's the statisticians fault (i.e. my fault) that the treatment effect is gone.

*Not enough power. Why even look at the data when just about half of the sample was collected?
Am I to overcautious or are this true concerns? What other problems might arise? Why and when would it be ok to look at the data prematurily? Would I have to adjust somehow the final analysis for having performed an interim analysis (assuming no changes in the design etc.)?
 A: I think that there is some confusion underlying this question. First of all, the answer very much depends what kind of analysis you're talking about - you're not clear on research design (despite mentioning of pre and post treatments) - observational study vs. experiment, true experiment vs quasi-experiment, with control group vs. without, whether you want/need to consider the effects of time (longitudinal design), whether you want/need to consider causal effects. For some exploratory data analysis (EDA) the Phase 1's sample size might not be a problem, but for some full-scale research designs even your combined sample might not be enough for valid statistical inferences.
Here are my thoughts on your particular concerns/reasons (I hope that this makes sense):


*

*You write: "the results of the interim analysis might affect the second phase". However, earlier you write: "the final analysis of the treatment effect is suppose to include all subjects from phase 1 and phase 2". It seems to me that there is a potential contradiction here, since if Phase 1 impacts Phase 2, how can you accurately account for that when doing final analysis, aggregating subjects from both phases? You might find helpful reading this chapter on treatment effects by Andrew Gelman.

*I agree with you on this concern. I would suggest collecting data from Phase 2 subjects, if you want to perform an aggregate analysis, that is use an aggregate sample.

*This is also a valid concern IMHO. Please see my points on the sample size in the beginning of this answer. However, nothing prevents you to assess the power of your potential analysis, using the Phase 1 sample size. But, even if it would be acceptable, the issues in #1 and #2 above might invalidate the results or conclusions anyway.
