2
$\begingroup$

this is my first post on CrossValidated. I've done, for academic purpose, a web tool doing this process:

  • web scraping from various sites
  • pre-process the responses (cleaning, error and redundance detection) and make a structured data out of these scraped informations (then I save these structures in db)
  • quantitative analysis and a sort of semantic analysis of the data
  • show interactive data visualization in different forms and purposes (node graph, line graph, treemap, so a sort of visual analysis)

Now I'm writing an article about the tool, but I'm in doubt about the terms explaining this process and its steps correctly (also some of those steps, the cleaning and error detection for example, some academic papers sustain it's part of data-mining, some other sustain it's a previous step).
From these few infos, my tool is more about information retrieval or data-mining?

Any advice about that (maybe some references too) would be appreciated.

$\endgroup$
2
$\begingroup$

The term data science is applicable here. I do this sort of work a lot, and although it doesn't contain very difficult statistical models, it gives your stakeholders new insights thanks to data. When you do a proper job and end up with a clean data set, your data will be so useful that you suddenly see opportunities to use them in models at the end.

Some more terms that I use:

  • data gathering for the scraping
  • data cleaning
  • data exploration for figuring out what you would like to show ( I would not use the term data mining here, it is not applicable )
  • dashboard, that is the most common term for the end product you are building ( although I dislike the term, lots of people are familiar with it. )

For a reference of terms, you can maybe check out the coursera classes of John Hopkins ( https://www.coursera.org/specialization/jhudatascience/1?utm_medium=spark_cdp_sidebar ) They have around 10 modules, each with a step of your process. The names of these modules might help you.

Hope this made sense / helped.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.