Thank you for joining me on this semi-theoretical journey. Here we will discuss how to account for "predictable" bias in your data.
Let's say we have a test composed on many subtests. A composite score is generated through successive aggregation and weighting of sub-tests.
Let's talk about sub-test A.
You can get a total score of 100 on sub-test A, and each item on the test has a different possible max score. Now you're missing (at-random) some answer for sub-test, rather than try to impute these scores, you can drop the missing items from the subtest and rescale.
How are we rescaling? Let's say you drop q10 and the MAX score you can get on this sub-test in 90. Multiply 90 by the NEW percent correct on this test to scale it back up. Now recalculate your composite score the same way as usual.
Now calculate the difference between your newly calculated/rescaled score and the original score in a sample of subjects who have COMPLETE sub-test sections and figure out how your data will be biased.
Here we are, when you use the method outlined above, we will on average skew the data by ~0.5 points if we drop the missing sub-items. Since I can measure this bias, would it be reasonable to then add back this bias as a constant in a linear regression?