There are many guides prevalent on the internet about EDA and how everyone should do it and how useful it is however I rarely see it in practice and often times (in said tutorials) it sticks to very basic things.
- Dimensions of data
- Plotting distributions of features
- Linear correlation among features
- Missing data (interpolating, dropping etc.)
I haven't often seen (with my limited sample size) that people actually do this in practice, especially on larger datasets where features range to hundreds-thousands, some of the above EDA techniques seem as more of a hinderance than help. Am I really expected to look at hundreds of plots of feature distributions for example?
I am not a formerly trained data scientist and I am still learning. I would like to add this tool to my toolkit, but aside from contrived examples on the internet, I have rarely found with real datasets that such techniques are useful to begin with. I normaly find myself in a circle, where I look a bit at my data, make some assumptions about what is useful and move on to modelling it. If / when something doesn't work, I normally have a better idea of which parts of the data to look at, saving me time when dealing with big datasets with hundreds of features.
If anyone can recommend a resource where I could improve my working / applied knowledge in this area it would also be appreciated. I realise this question is more of a soft question but I do feel it is important to clarify. I hope in its current format it can be seen as a question that can be given a definitive answer.