1
$\begingroup$

I have a question concerning the analysis of experimental data.

Imagine the following situation (which I simplify to make the point clearer):

  • I ask my participants to give a numerical answer to 3 questions, say, A, B, and C. So in total there are 3 data points per participant.

  • Now, I can compute several "expected" answers to question C according to competing theories. To simplify, imagine I consider theories X and Y, which yields two distinctive predictions for C: C-X and C-Y.

  • These "expected" answers are computed by combining the actual answers to questions A and B. For instance, let's say that theory X predicts that C equals A+B, while theory Y states that C should amount to the average of A and B (this is just an example, not the real case of my dataset).

  • Thus, note that my two "expected" predictions for C are based on a theory, but also on the actual answers given to A and B. This means that I have different predictions for each participant (since I also have different answers to A and B for each participant). And it also means that the predictions are not independent from the actual data of A and B (not the ones I want to model, i.e., answers to C).

  • Finally, note also that the theoretical predictions may correlate between them, since they are relatively similar computations on the same values (A and B).

My goal: As you may suppose by now, I would like to check whether the actual answers to C support theory X, or theory Y, or both, or none. A relative, rather than absolute, measure of support (e.g., "the evidence is 3 times more likely under theory X than uncer Y") would also be fine, but this sounds more like model selection and I'm not sure that this is what I want.

Would you help me decide the type of data analysis that suits better this situation? I can easily come up with simple solutions (e.g., a correlation model), but maybe they have drawbacks or there are more informative and sofistified alternatives... Any suggestion is welcome and I'm willing to learn!

EDIT: To further clarify, A, B, and C can be assumed continuous.

Improve this question
$\endgroup$
  • $\begingroup$ Am I correct in assuming that C is a continuous variable? If so, agreement can be indexed in terms of absolute deviation or in terms of correlation. Do you have a preference? (These two approaches can give different results.) Also, correlation can be linear or curvilinear. Do you have a prediction of what you will find? $\endgroup$ – Joel W. Sep 4 '12 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.