Please suggest a data entry tool (if one exists) for a single large sparse table.

Something like...

row x, checkbox list of 300 columns, check 5 that pertain, enter data for those 5.

row x+1, same checkbox, check different 14 that pertain, enter data for those 14.

row x+2, checkbox does not contain column, add column title and data type, check new column created and 33 others that pertain, enter data for those 34.

Although possible, it's inconvenient to do in a worksheet - too much tabbing/arrowing around.

  • 1
    $\begingroup$ I don't think there are ready made data entry tools for R. If you'd make an example data sheet and post it to StackOverflow (stackoverflow.com/questions/tagged/r), you could get some good suggestions how to handle your case. I am afraid that this kind of stuff is off-topic on CrossValidated... $\endgroup$ – Mikko May 7 '12 at 14:44
  • 2
    $\begingroup$ @Largh, Arguably, this question falls under the category of "collecting data," which is specifically on topic according to our FAQ. $\endgroup$ – whuber May 7 '12 at 21:44
  • $\begingroup$ @whuber. True. It's in an eye of the beholder, I guess. Probably not off-topic then. $\endgroup$ – Mikko May 8 '12 at 7:06
  • 1
    $\begingroup$ @kellyjeglum: Maybe you should make your question a bit more precise to get better answers than my stupid comments. Are you planning to collect the data or do you already have it? Is it a poll? Through which channel (internet, paper, etc.)? If you already have the data, I would suggest that you include an example of it. Something we can import to R. You can get instructions how to make it from here: (stackoverflow.com/questions/5963269/…) $\endgroup$ – Mikko May 8 '12 at 7:06

+1 for the question. I have not searched the web a lot for existing tools (presumably you did before posting your question here), but I am guessing someone would have to create a GUI to submit data like you want.

You need to consider what kind of format you want to work with elsewhere in your analysis though, because you may want to work with sparse formats in your code, but interact with the data as a traditional 2D matrix when viewing it or manually modifying elements. This question is tagged with R, but it is not clear from the question itself how you are using R. If this needs to be R, then ignore the rest of this...

If I had to create this functionality myself I would do it in Python using a few libraries

  • Create a GUI using formlayout to take data row/column entries with very little code
  • Take the user submissions and store them in a variety of sparse formats using scipy or pandas, which also let you easily go back and forth between sparse and dense formats.
  • Use tablib (or built-in methods for pandas data frames) to write data to Excel or other "tabular data" file formats. If you want to programmatically read/write/format Excel then consider using pytools or win32com
  • If you are working with the data at an interactive session, the Spyder IDE provides a great "variable explorer" GUI that you could use to manually modify elements of your densely-viewed-but-actually-sparse-format data.

So, hopefully someone posts an ready to go solution for you, but I do not think it would be terribly time consuming or difficult for you (or someone you know) to create what you need.

Good luck!


Software that is designed for "CATI" (computer assisted telephone interviewing) is usually very good for fast keyboard based data entry. "CfMC" is an example.

The way you could design this in a CATI-type data collection program would be to have one multi-select question with 300 options. The data entry operator would see a screen with all 300 options listed, and could punch in the numbers one at a time using a numeric keypad. For example, they would punch 12, 122, 150, 264, 299, and then continue.

For each of the 300 options, there would be a followup question conditioned to whether or not it was selected in the first question. So, if they selected five options on the first question, they would receive 5 followup screens one after another, and enter the values for those columns. From there it is very easy to get that sparse table into whatever format you like.

  • $\begingroup$ Actually as I re-read your question, this probably wouldn't support the action you need on row x+2. "add column title and data type". $\endgroup$ – Jonathan May 8 '12 at 19:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.