# Inferential Testing

I am currently working on my dissertation, which is investigating the relationship between working memory capacity and problem-solving ability. I have two independent controls task 1 and task 2, both measuring the same dependant controls (response time and accuracy).

i'm having troubles working out which kind inferential task to run. Either a correlation test, such as Pearson's product moment correlation or partial correlation. Or if i should be running a multiple regression test.

Any ideas?

Thank you

• Can't respond responsibly without more information. How many subjects? How many measurements of each kind (response time, accuracy) for each subject? Please show some data, or speculations of how data may look. Response time is typically more nearly exponential than normal. – BruceET Apr 28 at 18:57
• 128 participants, completing both conditions task 1 and task 2 with the response time (ms) and response accuracy being measured. It is speculated that there will be a strong correlation between task 1 and task 2 (those who perform better on task 1 will perform better on task 2). – Leah Apr 28 at 19:08
• Can you say how the data are intended to provide information about a connection between working memory and problem solving ability? If those are connected with Tasks 1 and 2, can you say how? In plain English, can you state your objective in terms of the two tasks? // Can you show the data for at least a few of the subjects? – BruceET Apr 29 at 1:59

OK, maybe that's enough information for a useful partial answer. It seems each of 128 subjects will have a performance score for Task 1 and Task 2, and you want to know if scores for one of the tasks run significantly higher than for the other.

Answering that requires a paired test. If the Task 1 minus Task 2 performance differences $$D_i$$ are roughly normally distributed, it would be a paired-t test, which is the same thing as a one-sample t test on the differences. [If data are nowhere near normally distributed, then there are other kinds of paired tests that can be used.]

For differences in performance on Tasks, I'd anticipate nearly normally data. For differences in response times (reaction times), I would not expect normal data. But one would have to look at the data to decide. I have done some work in testing reaction times to various stimuli and I have never seen reaction times anywhere near normal. [Medians of moderately large numbers (20 or 30) of reaction times can be nearly normal.]

Two general observations on paired data:

• If the two tasks are at all similar, then of course, you'd expect a positive correlation between Task 1 and Task 2 scores over the 128 subjects. That is characteristic of most paired data.

• I hope not everyone does Task 1 before Task 2. Otherwise, 'practice' or 'familiarization' with the equipment or evaluation mechanism will be 'confounded' with the difference between Tasks. It would be best to have half the subjects do Task 1 first and the other half do Task 2 first. That way you could test whether there is an order effect and, separately, test whether there is a difference between Tasks.

Illustrations with fake normally-distributed, paired data: The first $$6$$ of $$n=128$$ values of $$T_1, T_2,$$ and $$D = T_2 - T_1$$ are shown below; for these, we see that $$T_2$$ tends to be greater than $$T_1.$$

head(cbind(t1, t2, d))
t1    t2     d
[1,]  86.3  79.4  -6.9
[2,] 102.6 107.7   5.1
[3,] 109.2 125.4  16.2
[4,]  63.0  68.6   5.6
[5,] 114.6 100.2 -14.4
[6,] 102.9 114.3  11.4


A stripchart of the 128 differences $$D_i$$ with $$\bar D = 3.80$$ marked with a vertical bar.

stripchart(d, pch="|");  abline(v=mean(d), col="red", lwd=2) A scatterplot of $$T_{2i}$$ (vertical) against $$T_{1i},$$ correlation $$r_{12} = 0.601:$$

plot(t1, t2, pch=20);  cor(t1, t2)
 0.6011453 Results of a paired t test, show that the Task 2 scores are significantly larger than Task 1 scores (P-value $$0.4\%):$$

t.test(t1, t2, pair=T)

Paired t-test

data:  t1 and t2
t = -2.9465, df = 127, p-value = 0.003825
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-6.349400 -1.247475
sample estimates:
mean of the differences
-3.798437             # Mean diff is negative...
# ... because t1 listed first.