Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 1 down vote favorite

I'm very new at statistics, so excuse me if my questions are to simple or obvious.

I have a dataset with response times for two conditions. These response times were converted into a binary score of either 0 (absence of the effect) or 1 (presence of the effect).

The experiment is designed in 7 blocks. For each block participants receive a 1 or 0. The total score for each subject can range from o to 7.

So, I want to do a split-half reliability to see if the criteria I used to evaluate the presence of the effect is a reliable indicator of what I'm studying. At first I just split the column of 1s and 0s in half and correlated it, but that give me a nonsense value (something like 0.0009) so I'm missing something here. Conceptually I know I should correlate two parts of the test that are supposedly measuring the same thing, but I'm having difficulties at figuring out exactly how to do this for this dataset.

Any help is much appreciated.

share|improve this question

2 Answers

up vote 0 down vote accepted

You need to split the test in half, i.e. in a group of 3 and a group of 4 items and then correlate those groups. So as if you have 2 tests both consisting of half the total test.

share|improve this answer
Thanks for your answer simmons. This seems a bit counter intuitive for me. Are you saying I would be correlating all scores obtained from block 1 to 3, on the one hand, and blocks 4 to 7 on the other? I would think the scores would naturally change as blocks go by, so even if the test is reliable I wouldn't expect results from the earlier blocks to be correlated with the results of the last blocks. – Hernan_L Jun 8 '12 at 16:06
The idea is that the test measures 1 dimension. So if you split it in half and correlate the them you get a measure of the consistency of the test, which gives you information if it really measures one thing. Edit: what do you mean with 'blocks go by'? Also you should randomly assign items into groups. Furthermore, you were asking for split-half reliability, which is exactly what I explained. If you want to do something else, please elaborate on what your goals are. – simmmons Jun 8 '12 at 17:54
Thanks simmons. Split-half reliability is indeed what I intended to do. My data corresponds to a motor learning task, where learning is measured by decreases in response time. I think what I have been doing wrong is that I was just splitting my data.frame in two halves and then correlating the two vectors of 1s and 0s... If you say I have to assign what goes into each half randomly, then I think I'm missing something. The relevant parts of my data.frame for this problem is one column with subjects and one with the 1 or 0 scores, and one column for block. How would you recommend to proceed? – Hernan_L Jun 8 '12 at 18:27
I am still not 100% sure what your experiment is, but I think I understand your data. You have for every subject 7 entries specifying subject id, score, and block id. The blocks are added together to get a compound measure, i.e. test score. What you called blocks, I called item. Now randomly assign items to groups and calculate the score for each subject and in both groups. You should now have three vectors: subject id, group1 and group2 score. Each subject id should occur only once! Then you correlate the score of group 1 with the score of group 2. – simmmons Jun 8 '12 at 18:48
Thanks for your clarifying answer simmons. Given my n is 219, I should have a column with 219 rows, one for each subject. What I don't understand is what goes in the other two columns. Suppose I were to assign to one half blocks 1,3,5 and to the other half blocks 2,4,6 and 7. Then in my first set of rows I would have subject one, the score he got on block 1 and the score he got on block 2. In the second row I would have subject 2, the score he got in block 3 and the score of block 4. In the third row subject 3 and the score he got in block 5 and then on block 6, etc. Is that it? – Hernan_L Jun 8 '12 at 19:37
show 3 more comments
up vote 0 down vote

It really sounds like you want to do cross-validation. I am assuming you use the data to construct the criterion. A way to do this that is a type of cross validation would be to randomly split the data into tow groups ( could be an equal split but you may want a larger sample for the fitting of the criterion). Construct the criterion on the first set and then test how it does.

share|improve this answer
Thanks for your suggestion Michael, but right now I'm trying to replicate some results, so I need to perform a split-half reliability test, even though there might be better ways to test the reliability. – Hernan_L Jun 8 '12 at 16:55
I don't know what that is or why it is approapriate. – Michael Chernick Jun 8 '12 at 17:24

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.