2
$\begingroup$

I have a cognitive architecture solving a set of tasks. Also I have data of human subjects solving the same set of tasks.

Now I want to see whether I can find a relationship between the performance of the architecture and that of human subjects (e.g: if its hard for the architecture its also hard for human subjects).

I have 3 summary statistics describing the performance of the architecture, call them a,b,c. They all measure slightly different things. Think about: uncertainty, structure of the solution given etc. For the human data I only have access to one statistic (data taken from a published experiment ), which is related but not identical to a,b,c.

If I compute the correlation coefficient, I see that b shows a high correlation to the performance of human subjects. However a,b,c have correlation (~0.2-0.7) in themselves.

I read that one can compute the partial correlation coefficient which measures the relationship of two variables excluding the influence of the other. If I do this, c has the highest partial correlation to human data, although its correlation to the human data was the lowest

I have trouble interpreting these results, and I am not sure whether I should do my interpretation based on correlation or partial correlation and if the use of partial correlation makes sense in this context. Also note: this is purely explorative, I don't aim to do hypothesis testing, I merely search for relationships.

$\endgroup$
4
  • $\begingroup$ It's unclear to me from your wording just what is being considered a dependent variable vs. an independent variable. And it's hard for me to see how architecture can "perform" or "have difficulty." But as I understand you, you are wise to consider partial correlation (mediation effects), and you also need to consider statistical interaction (moderation effects).... $\endgroup$ – rolando2 Jul 12 '14 at 17:14
  • $\begingroup$ ...This means you'll want to move from bivariate analysis to a multiple regression or ANOVA model. Whenever the relationship between the outcome and one predictor changes depending on the level of another predictor, one needs to assess the strength and direction of an interaction between the two predictors. You can find more on testing, assessing, and interpreting interactions by doing a search for that term on this site. $\endgroup$ – rolando2 Jul 12 '14 at 17:15
  • $\begingroup$ Thanks for the answer! To clarify: suppose a,b,c (sumary statistics) are the independent and d (human performance) is the dependent variable. Wouldn't it make more sense to look at the semi-partial correlation then? E.g for statistic a to calculate out the effect of b,c but not for the dependent variable. $\endgroup$ – Stefan Jul 12 '14 at 18:51
  • $\begingroup$ Some background: given are a set of logic puzzles. The model approximates a posterior distribution over logical formulas,correct solutions to these puzzles are only given in natural language in the literature. Human experimental data is #Correct/#N for each puzzle (no more descriptions). I take measures from the posterior: entropy, length of MAP solution, score (how 'correct' the most probable formula is which sometimes is debatable). Now I want to see for example how much the entropy or score is related to the human performance. $\endgroup$ – Stefan Jul 12 '14 at 18:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.