0
$\begingroup$

I have a binary classification prediction task and more often than not, the time series data is like really sparse. The number of zeroes in the time series data is almost always more than 99%. I believe this is causing my RNN and LSTM models to behave really weird and unstable. The validation scores are highly variable between multiple runs and there there are lot of ties when it comes to class probability predictions.

The time series data is derived based on some dates. It somewhat looks like below: enter image description here

The above data is showing just 2 tenants. Tenant 9 had encountered two items on 2009-09-11 and 2009-09-26. Each row in item_id is a tuple of length N where N is the number of distinct item_id for all tenants. So if N is 100, the tuple corresponding to 2009-09-11 for tenant 9 will have all zeroes except one location where it will be 1 for the item encountered. Now you can imagine how sparse this will be, especially after padding! In some cases its almost 100% sparse. Highly doubt this will be useful for any sorts of modeling.

My models are very unstable even after training for large number of epochs and I notice there are ties on a range of 20 to 70% - meaning the predictions are inconclusive in distinguishing between the classes.

Questions:

  1. Is the sparseness really the reason for models being so unstable and inconclusive?

  2. One way to deal with the sparseness is to make the data less granular by deriving time sequence using month and year or year instead of the actual date. Is this an approach worth trying?

$\endgroup$
3
$\begingroup$
  1. Is the sparseness really the reason for models being so unstable and inconclusive?

Yes. At 99% zeros, just about any time series method is going to fail, especially one as "data hungry" as RNN or LSTM. You're better off modeling it as a Poisson process.

  1. One way to deal with the sparseness is to make the data less granular by deriving time sequence using month and year or year instead of the actual date. Is this an approach worth trying?
  • The granularity of your predictions is usually driven by your business objective, not just your data.

  • Using Temporal aggregation to improve forecasts for intermittent demand was suggested by Petropoulos and Kourentzes in "Forecast combinations for intermittent demand". In your case, that means that you have on average less than 1 non zero data point every 100 days, so you could aggregate up to a quarterly level (1 quarter = 3 months).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.