A friend of mine has suggested that the software industry primarily
needs "big data" skills, not statistics skills per se.
While partially agreeing with your friend's comment, I would like to point out that in any industry, Big data tools are opted, only if all the V's are satisfied.
I work as the head of data science at a leading customer support company. Here, I do data hacking both for the product and also for the growth of the company.
I primarily use time series analysis techniques for churn prediction and sales analysis. This also includes the behavioural analysis of the customers, competition and the industry.
On the product side, we use a range of techniques starting from sentiment analysis using LSTM's, recommendation algorithms, etc.
But the core focus lies on time series analysis. The general workflow would be:
- Cleaning and moulding the data.
- the exploratory and explanatory analyses which involves identification of seasonality, trends and cycles. So, one need to explore correlations, auto-correlations, and several univariate and bivariate statistics; along with extensive plotting including the scatter, AFC, PAFC curves.
- Now comes the forecasting part, where various models are tested each other, taking the step - 2 into serious consideration.
Tools used by me: R, Python and Excel sometimes.
And even the blend of data science and growth hacking have proven to do magic in the domain of marketing. So, the demand for statisticians and math nerds would remain as is; and is not going to decline anywhere in the near future; especially when customer focused startups are blossoming across the world.