Assessing effect of intervention where aptitude tests used in pretest and posttest have some overlapping and some distinct items In my research there is a pretest, and a treatment followed by a posttest. 
The pre and the posttests are similar but not exactly the same. A few questions were changed between the pre and posttests to avoid sensitizing the students to the questions in the pre-test, which might affect the internal validity of the study. 
Because I changed the post test, I got into the instrumentation threat, because now the pre and post tests are not exactly the same. In order to eliminate the instrumentation threat, I plan to compare the pre and the posttests taking only  the common questions from both tests.


*

*Is this permissible statistically? 

*Is there any literature that supports or talks about unequivalent pre and post tests? 

 A: Do you have equivalent tests anyway?


*

*If you are only analysing the common questions from pre- and post- then under most circumstances you now have equivalent tests. 
Possible exceptions to this idea might occur if test items weren't independent (e.g., you had some unique difficult questions at the start of one test that meant  that participants had less time for the remaining test items).

*Hopefully you have enough common items for  the test to still be reliable with the smaller number of items. You can check this by doing a reliability analysis on the abbreviated test.


Confounds and design issues


*

*Even with equivalent tests, practice effects are still a potential confounding interpretation of any change.

*Your design would be strengthened if you had a control group that did not receive the intervention. In particular, without a control group, it is difficult to say what the effect of the intervention is over and above any general effects of time.


Designing equivalent forms
There are many instances of test publishers developing equivalent forms of tests. 


*

*An example of the development of an equivalent form is: Huff, Collins, and Rosen (1986) Equivalent Forms of the Boston Naming Test. Journal of Clinical Experimental Neuropsychology, 8, 556-562. PDF

*Computerised adaptive testing is another option that you could consider in the future.

*Cook, L.L. and Eignor, D.R. (1991). IRT equating methods. Educational Measurement: Issues and Practice, 10, 37-45. PDF This is a tutorial on using IRT methods to develop equivalent forms of ability tests.


Designing equivalent forms does require care, and is not simply a matter of modifying a few items.
Missing data / data imputation


*

*You might consider treating your situation as a missing data problem. Thus, the pre and post tests could be thought of as made up of three scores based on three sets of items: common, unique to time 1, and unique to time 2. The problem is that you have missing data at time 1 for items unique to time 2, and at time 2 you have missing data on items unique to time 1.

*I've never done this form of imputation (repeated measures with highly structured and systematic missing data), but I imagine there would be some form of data imputation procedure that took advantage of the available information to impute the missing data. You could then calculate imputed means for time 1 and time 2 based on all three sets of scores.

