I am not sure whether this question is fully appropriate here, if not, please delete.
I am a grad student in economics. For a project which investigates issues in social insurances, I have access to a large number of administrative case reports (>200k) which deal with eligibility evaluations. These reports can possibly be linked to individual administrative information. I want to extract information from these reports that can be used in quantitative analysis, and ideally more than simple keyword/regex searches using grep
/awk
etc.
How useful is Natural Language Processing for this? What are other useful text-mining approaches? From what I understand this is a large field, and most likely some of the reports would have to be transformed to be used as a corpus. Is it worth investing some time to become acquainted with the literature and methods? Can it be helpful and has something similar been done before? Is it worth it in terms of the rewards, i.e. can I extract potentially useful information using NLP for an empirical study in economics?
There is possibly funding to hire somebody to read and prep some of the reports. This is a larger project and there is a possibility to apply for more funding. I can provide more details about the topic if strictly necessary. One potential complication is that the language is German, not English.
Regarding qualifications, I am mostly trained in econometrics, and have some knowledge about computational statistics at the level of the Hastie et al. book. I know Python, R, Stata, and could probably get familiar with Matlab quickly. Given the libraries, I assume Python is the tool of choice for this. No training at all in qualitative methods if this is relevant, but I know some people I could reach out to.
I am glad for any input on this, i.e. if this is potentially useful, if so, where to start reading and which tools to focus on in particular.