Timeline for Strategy for editing comma separated value (CSV) files
Current License: CC BY-SA 2.5
12 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 8, 2010 at 16:11 | comment | added | russellpierce | I glanced at Wikipedia's page, and yes DIF files are plain text... but they also look more computer readable than human readable and seem like they'd also be difficult to do data entry with. | |
| Dec 8, 2010 at 10:44 | comment | added | Jeromy Anglim | @Gavin Thanks for the suggestion about Data Interchange Format. I've asked a separate question about this: stats.stackexchange.com/questions/5249/… | |
| Dec 8, 2010 at 10:35 | history | edited | Gavin Simpson | CC BY-SA 2.5 |
make clear what is update and what was original; added 14 characters in body
|
| Dec 8, 2010 at 10:04 | history | edited | Gavin Simpson | CC BY-SA 2.5 |
Add note on DIF files
|
| Dec 8, 2010 at 9:05 | comment | added | Jeromy Anglim | @Gavin I agree about having a script that processes a supplied data file. I guess I'm thinking about some relatively ad hoc files that I create (for example, I might create a little csv file that lists the variable names and corresponding labels that I'll use when preparing an R table). | |
| Dec 8, 2010 at 9:05 | comment | added | Gavin Simpson | @Jeromy: re: losing numeric formatting - yes, it will, unless you specify the column type as "numeric" (Gnumeric) or "text" in (OpenOffice.org) upon import. Better might be to store it as text (see my edited answer) to avoid the conversion - you'll still need to specify the data type upon import... | |
| Dec 8, 2010 at 9:02 | history | edited | Gavin Simpson | CC BY-SA 2.5 |
some extra clarifications
|
| Dec 8, 2010 at 8:58 | comment | added | Gavin Simpson | @Jeromy: fair point on the warnings. I've long since filtered them out as noise so they don't bother me. I just tested Gnumeric and OpenOffice.org and they convert a csv with 12/3 to dates automagically - that is rubbish! So I see what you mean. The only way to stop that would be to store those data as text and force that data type on load/import. | |
| Dec 8, 2010 at 8:50 | comment | added | Gavin Simpson | @Jeromy yes, it would, if you were strict about making a single change (or single data processing step) that you immediately save and commit. I mean't to include a note about doing the processing in R (as it is my preferred language) using a script. That is how I work with data colleagues send me; I might have a quick look in a spreadsheet, then read the csv into R and write a script that includes all the data processing steps I need to apply. I then comment that script so I have a record of what I did and why I did it and I haven't changed the original data file at all. | |
| Dec 8, 2010 at 8:48 | comment | added | Jeromy Anglim | with regards 12/3, I had an example of a multiple choice test where this represented the 12 divided by 3. However, I use it more as an illustrative example. When I have a large CSV file, I don't want even one inappropriate conversion. With regards to warnings, If I'm opening CSV files 10 times a day the warnings get tedious. They also make it less clear whether I have actually saved the CSV file or not. Sorry for the rant. | |
| Dec 8, 2010 at 8:45 | comment | added | Jeromy Anglim | With regards to reproducible research, if the file format is CSV, then the CSV file could easily be put under version control with commits when changes are made. Would this suffice for reproducible research? With regards to data entry, I often do or get others to do raw data entry in fix width format in order to maximise efficiency; however I prefer to store data in CSV or TSV (I find it more transparent); with regards to formatting a column as numeric. this information would presumably be lost on saving. | |
| Dec 8, 2010 at 7:31 | history | answered | Gavin Simpson | CC BY-SA 2.5 |