I have a dataset of time series that present popularity of words (or phrases). Each words is a list of frequencies according to a timestamp. My purpose is to detect a specific type of spike that leads to an event A. For example in my data I know that some spikes in the words lead to an event A while other spikes of similar frequency did not. Currently I am using a custom method measuring the height of the peak, the increase ratio relative to the baseline activity, and existence of some symbols in the words (or phrases). I am using custom thresholds in order to make decisions. I know that this is not the right way to do that, for example I could use an optimizer to find the best thresholds apart from trial and error.
So I decided to model only the peaks of the time series and create a dataset of peaks that lead to event A and peaks that lead to event B. Using the visualizations it is very difficult to understand the difference in the peaks between the two types.
Now my question is:
- Is this procedure the right one for such a problem?
- How can I represent a peak? What kind of features can I use? I am thinking of using my current thresholds as features (frequency,growth ratio,decay ratio) but I am not sure if this is appropriate.
- I am thinking of using these features to train a logistic regression classifier or a neural network. Is this straitforward?
My dataset creation is based on Grubbs test for outliers. When ever I see an outlying word this means that there is a peak. I store the peak with the above information and its class label (A or B).
Since using the visualizations I cant understand the difference between the type of peaks means that my features are not informative. What relevant problems can I look for to have an idea?