Data entry tool for sparse table

Question

Please suggest a data entry tool (if one exists) for a single large sparse table.

Something like...

row x, checkbox list of 300 columns, check 5 that pertain, enter data for those 5.

row x+1, same checkbox, check different 14 that pertain, enter data for those 14.

row x+2, checkbox does not contain column, add column title and data type, check new column created and 33 others that pertain, enter data for those 34.

Although possible, it's inconvenient to do in a worksheet - too much tabbing/arrowing around.

Answer 1

+1 for the question. I have not searched the web a lot for existing tools (presumably you did before posting your question here), but I am guessing someone would have to create a GUI to submit data like you want.

You need to consider what kind of format you want to work with elsewhere in your analysis though, because you may want to work with sparse formats in your code, but interact with the data as a traditional 2D matrix when viewing it or manually modifying elements. This question is tagged with R, but it is not clear from the question itself how you are using R. If this needs to be R, then ignore the rest of this...

If I had to create this functionality myself I would do it in Python using a few libraries

• Create a GUI using formlayout to take data row/column entries with very little code
• Take the user submissions and store them in a variety of sparse formats using scipy or pandas, which also let you easily go back and forth between sparse and dense formats.
• Use tablib (or built-in methods for pandas data frames) to write data to Excel or other "tabular data" file formats. If you want to programmatically read/write/format Excel then consider using pytools or win32com
• If you are working with the data at an interactive session, the Spyder IDE provides a great "variable explorer" GUI that you could use to manually modify elements of your densely-viewed-but-actually-sparse-format data.

So, hopefully someone posts an ready to go solution for you, but I do not think it would be terribly time consuming or difficult for you (or someone you know) to create what you need.

Good luck!

Answer 2

Software that is designed for "CATI" (computer assisted telephone interviewing) is usually very good for fast keyboard based data entry. "CfMC" is an example.

The way you could design this in a CATI-type data collection program would be to have one multi-select question with 300 options. The data entry operator would see a screen with all 300 options listed, and could punch in the numbers one at a time using a numeric keypad. For example, they would punch 12, 122, 150, 264, 299, and then continue.

For each of the 300 options, there would be a followup question conditioned to whether or not it was selected in the first question. So, if they selected five options on the first question, they would receive 5 followup screens one after another, and enter the values for those columns. From there it is very easy to get that sparse table into whatever format you like.