Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 5 down vote favorite

I am trying to assess a 20-item multliple choice test. I want to perform an item analysis such as can be found in this example. So for each question I want the P-value and the correlation with the total, and the distribution of the options selected.

I don't know anything about the various statistical software packages out there, but I'd like to use R as I'm comfortable with programming and R is open source. The pseudo-workflow I envision is:

  1. prepare data in excel and export to CSV

  2. load data in R

  3. load a package that does what I need

  4. execute that package's commands

  5. export and report.

I am confident with 1 and 2 but having trouble with 3, probably because I don't have the statistical vocabulary to compare the packages I browsed on CRAN. ltm looks like it could be the right package but I can't tell. Whatever package is used, what would the commands be?

Side question: in the linked example, what do you suppose MC and MI stand for?

share|improve this question

2 Answers

up vote 6 down vote

I can suggest you at least two packages that allow to perform these tasks: psych (score.items) and ltm (descript). The CTT package seems also to process MCQ but I have no experience with it. More information can be found on W Revelle's website, The Personality Project, esp. the page dedicated to psychometrics with R which provides step-by-step instructions for importing, analyzing and report data. Also, the CRAN Task View on Psychometrics includes many additional resources.

As described in your link, MC stands for "Mean total raw score of the persons who answered the item with the correct response", and MI for "Mean total score of the persons who did not answer the item with the correct response.". Point-biserial correlation (R(IT)) is also available in the ltm package (biserial.cor). This is basically an indicator of the discrimination power of the item (since it is the correlation of item and total score), and is related to the discrimination parameter of a 2-PL IRT model or factor loading in Factor Analysis.

If you really want to reproduce the table you show, I guess you will have to wrap some of this code with custom code, at least to output the same kind of table. I've made a quick and dirty example which reproduce your table:

dat <- replicate(10, sample(LETTERS[1:4], 100, rep=TRUE))
dat[3,2] <- dat[67,5] <- NA
itan(dat)

         P      R    MC    MI NC OMIT  A  B  C  D
 [1,] 0.23 -0.222 2.870 2.169 23    0 23 22 32 23
 [2,] 0.32 -0.378 3.062 1.985 32    1 32 20 14 33
 [3,] 0.18 -0.197 2.889 2.207 18    0 18 33 22 27
 [4,] 0.33 -0.467 3.212 1.896 33    0 33 18 29 20
 [5,] 0.27 -0.355 3.111 2.056 27    1 27 23 23 26
 [6,] 0.17 -0.269 3.118 2.169 17    0 17 25 25 33
 [7,] 0.21 -0.260 3.000 2.152 21    0 21 24 25 30
 [8,] 0.24 -0.337 3.125 2.079 24    0 24 32 22 22
 [9,] 0.13 -0.218 3.077 2.218 13    0 13 29 33 25
[10,] 0.25 -0.379 3.200 2.040 25    0 25 25 31 19

As these are random responses, biserial correlation and item difficulty are not very meaningful (except to check that data are truly random :). Also, it is worth checking for possible errors, since I drafted the R function in 10'...

share|improve this answer
1  
Thanks for this! I'm reading your code and it's helping me figure out the R syntax a lot. One thing that's giving me trouble is the line freq.resp <- raw.resp/apply(raw.resp, 1, sum, na.rm=T). The error is that "dim(X) must have positive length" while dim(raw.resp) is NULL. Could it be that since my data doesn't have all options with positive frequencies, my tables aren't all the same length? How can I fill in the zeroes in my table invocation? – Matthew Leingang Oct 30 '10 at 12:03
@Matthew Yes, it is very likely that this is indeed the source of the problem (well, you're learning R quickly, I appreciate). So, if one response category is not observed for one or more items, then the length of the table will be < 4, and freq.resp will throw an error. I will fix the code (by tomorrow), this will also show you how to concatenate irregular tables with little code. – chl♦ Oct 30 '10 at 12:47
@Matthew I've added a fix. Should work with dichotomous items too. Let me know if this is now ok. gist.github.com/642219 – chl♦ Oct 31 '10 at 9:04
Thanks so much for holding my hand. Your function gives me an error: "Error in raw.resp[i, names(tmp)] <- tmp : subscript out of bounds", but oddly enough, I could run each line of the function without problems. So I have a working report. – Matthew Leingang Nov 1 '10 at 12:29
up vote 4 down vote

The psych package function alpha produces what you are looking for.

To export this, save the object and use the xtable function to produce LaTeX markup, which can then be formatted by any LaTeX editor.

share|improve this answer
(+1) psych::alpha does a better job than score.items, thanks for adding that. I never remember the one I should use! – chl♦ Oct 23 '10 at 13:53

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.