When I work on data analysis projects I often store data in comma or tab-delimited (CSV, TSV) data files. While data often belongs in a dedicated database management system. For many of my applications, this would be overdoing things.

I can edit CSV and TSV files in Excel (or presumably another Spreadsheet program). This has benefits:

• spreadsheets makes it easy to enter data

There are also several problems:

• Working with CSV and TSV files leads to a wide range of warning messages about various features being lost and how only the active sheet will be saved and so forth. Thus, it's annoying if you just want to open the file and make a little change.
• It does many "supposedly intelligent" conversions. For example, if you enter 12/3, it will think that you want to enter a date. UPDATE: I should have mentioned that the date example is just one of many examples; most problems seem to be related to inappropriate conversion. In particular, text fields that look like numbers or dates cause problems.

Alternatively, I could work directly with the text file in a standard text editor. This ensures that what I enter is what is recorded. However it is a very awkward way to enter data (columns don't line up; it's difficult to enter data simply into multiple cells; etc.).

Question

• What is a good strategy for working with CSV or TSV data files? i.e., what strategy makes it easy to enter and manipulate the data while also ensuring that what you enter is actually interpreted correctly?

When I work on data analysis projects I often store data in comma or tab-delimited (CSV, TSV) data files. While data often belongs in a dedicated database management system. For many of my applications, this would be overdoing things.

I can edit CSV and TSV files in Excel (or presumably another Spreadsheet program). This has benefits:

• spreadsheets make it easy to enter data

There are also several problems:

• Working with CSV and TSV files leads to a wide range of warning messages about various features being lost and how only the active sheet will be saved and so forth. Thus, it's annoying if you just want to open the file and make a little change.
• It does many "supposedly intelligent" conversions. For example, if you enter 12/3, it will think that you want to enter a date. UPDATE: I should have mentioned that the date example is just one of many examples; most problems seem to be related to inappropriate conversion. In particular, text fields that look like numbers or dates cause problems.

Alternatively, I could work directly with the text file in a standard text editor. This ensures that what I enter is what is recorded. However it is a very awkward way to enter data (columns don't line up; it's difficult to enter data simply into multiple cells; etc.).

Question

• What is a good strategy for working with CSV or TSV data files? i.e., what strategy makes it easy to enter and manipulate the data while also ensuring that what you enter is actually interpreted correctly?

When I work on data analysis projects I often store data in comma or tab-delimited (CSV, TSV) data files. While data often belongs in a dedicated database management system. For many of my applications, this would be overdoing things.

I can edit CSV and TSV files in Excel (or presumably another Spreadsheet program). This has benefits:

• spreadsheets makes it easy to enter data

There are also several problems:

• Working with CSV and TSV files leads to a wide range of warning messages about various features being lost and how only the active sheet will be saved and so forth. Thus, it's annoying if you just want to open the file and make a little change.
• It does many "supposedly intelligent" conversions. For example, if you enter 12/3, it will think that you want to enter a date.

Alternatively, I could work directly with the text file in a standard text editor. This ensures that what I enter is what is recorded. However it is a very awkward way to enter data (columns don't line up; it's difficult to enter data simply into multiple cells; etc.).

Question

• What is a good strategy for working with CSV or TSV data files? i.e., what strategy makes it easy to enter and manipulate the data while also ensuring that what you enter is actually interpreted correctly?

When I work on data analysis projects I often store data in comma or tab-delimited (CSV, TSV) data files. While data often belongs in a dedicated database management system. For many of my applications, this would be overdoing things.

I can edit CSV and TSV files in Excel (or presumably another Spreadsheet program). This has benefits:

• spreadsheets makes it easy to enter data

There are also several problems:

• Working with CSV and TSV files leads to a wide range of warning messages about various features being lost and how only the active sheet will be saved and so forth. Thus, it's annoying if you just want to open the file and make a little change.
• It does many "supposedly intelligent" conversions. For example, if you enter 12/3, it will think that you want to enter a date. UPDATE: I should have mentioned that the date example is just one of many examples; most problems seem to be related to inappropriate conversion. In particular, text fields that look like numbers or dates cause problems.

Alternatively, I could work directly with the text file in a standard text editor. This ensures that what I enter is what is recorded. However it is a very awkward way to enter data (columns don't line up; it's difficult to enter data simply into multiple cells; etc.).

Question

• What is a good strategy for working with CSV or TSV data files? i.e., what strategy makes it easy to enter and manipulate the data while also ensuring that what you enter is actually interpreted correctly?