I try to produce test/retest coefficient for our computer based executive function test using intraclass correlation coefficient. The test is very structured computer based test and it has hundreds of short Go/Nogo trials. The subjects have executed the test twice few weeks apart each test.

So I have dataframes consisting all single trials(df1) and dataframe (df2) consisting average reaction times per subject (n=20) and error percentages per subject.

Question 1: The fixed model is the right model to use, I suppose?

Question 2: What is the true difference between average and single measures at this kind of situations? Which is more recommendable to use, ICC(3,1) or ICC(3,k)?

I suppose using the use of df2 and ICC is best way to evaluate the repeability of our test.

Thank you!

I try to produce test/retest coefficient for our computer based executive function test using intraclass correlation coefficient. The test is very structured computer based test and it has hundreds of short Go/Nogo trials. The subjects have executed the test twice few weeks apart each test.

So I have dataframes consisting all single trials  (df1) and dataframe (df2) consisting average reaction times per subject ($n=20$) and error percentages per subject.

Question 1: The fixed model is the right model to use, I suppose?

Question 2: What is the true difference between average and single measures at this kind of situations? Which is more recommendable to use, ICC(3,1) or ICC(3,k)?

I suppose using the use of df2 and ICC is best way to evaluate the repeatability of our test.