I need to perform and evaluate predictions on a time series with some 3,000 categories as independent variable. Specifically, I have 3 fields:

  • "Value", which is my independent variable (the one that I'd like to predict)
  • "category": it has some 3,000 different levels, in the form of Integers:example, in the snapshot here below, 2237 and 1497 are two levels of the 3,000 levels of the "category" variable.
  • "Date": self explained (see below)

For each date, I have around 3,000 records, one for every level of the "category" variable.

It's actually my only independent variable to explain the time series (if I don't use the "Date" field). Which method should I use?

Date        Value   category
2017-10-01  9748.0  485
2017-10-01  0.0     1242
2017-10-01  1706.0  14
2017-10-01  50.0    1001
2017-10-01  0.0     2235
2017-10-01  0.0     1497

A first thought would have been to fit an ARIMA model for each category... but I am wondering if there is a more synthetic approach.

Thanks in Advance for your precious advices!

  • $\begingroup$ You could use a limited number of group of levels. If the factor is ordered, then it could be transformed into a continuous variable using a parameterised transformation. With the 'groups' solution you can have a time series model for each group and correlation within groups. $\endgroup$ – Yves Feb 1 at 12:41
  • $\begingroup$ Thanks @Yves for your answer. The factor is not ordered. Can you elaborate on this please? "then it could be transformed into a continuous variable using a parameterized transformation" $\endgroup$ – Ronicho Feb 1 at 13:54
  • $\begingroup$ If your factor was ordered, then you could map the $m$ ordered levels to a regular sequence e.g. $1$, $\dots$, $m$ and then apply a transformation such as a normal cdf which would give you a variable $u_t$ which could be used as a covariate. With a high number of levels a more flexible transformation such as a non-decreasing spline with $k$ knots could be used. $\endgroup$ – Yves Feb 1 at 14:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.