3
$\begingroup$

I am forecasting the spending of customer based on the time series data set. I tried to build to models:

  • Arima
  • regression. ´

With Arima model, I was able to forecast quite accurately the spending of customers on normal day. However, in both my data set and present reality, on every first day of month, due to monthly promotion, the customers' spending is always significantly higher than that on other days. Hence, with Arima models, at the moment I have not been able to forecast the customers' spending on first day of month accurately. With regression model, the predicted number for the customers' spending on first day of month was more accurate than that predicted with Arima model.

However, with an $R^2$ of only 50%, the fluctuation of predicted values for others days is very high. Can you please advise me methods that I can use to improve my models or is there any other models that can help me to predict the spending accurately on both normal days and first day of month? Here is a small part of my data enter image description here

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ use the xreg option of arima to add a binary indicator for the first day of the month. $\endgroup$ – Harlan Nelson Apr 13 '18 at 3:29
  • $\begingroup$ Hi Nelson, I have been trying your method and it did work. However, there are still something went wrong which made the results not really perfect. Would you mind advising more for my case? Thank you very much! $\endgroup$ – Nguyen Ba Viet Apr 16 '18 at 10:21
  • $\begingroup$ @Harlan Nelson: I did use the xreg option as you suggested and ended up with this coding: library(MASS) library(timeSeries) library(forecast) df <- read.csv("C:/Users/ADMIN_VP/Desktop/ts-tdg.csv", header = TRUE) #file updated daily tstdg <- ts(df$TDG) fit <- auto.arima(tstdg, xreg=df$S_F_day_ofmon) fit forecast(fit, xreg=df$S_F_day_ofmon). There are 2 problems here: 1. I only want to forecast the next 5 days but forecast function only works if I put xreg as above $\endgroup$ – Nguyen Ba Viet Apr 17 '18 at 1:54
  • $\begingroup$ 2. the code seemed to work if my data ended last day of month (which the next forecast will be for first day of month). However, when I deleted some rows to test whether it forecasts well on other days, the first result is still showing like it is forecasting for first day of month (which is much higher than spending on other days). I tried to add seasonal to my model as well but had to reduce all months to 28 days because numbers of days in each month are different (28, 29, 30 and 31). Can you please advise me on how I can improve my model? Many thanks! $\endgroup$ – Nguyen Ba Viet Apr 17 '18 at 1:57
  • $\begingroup$ To share the data, use e.g. sharecsv.com $\endgroup$ – Jim Apr 19 '18 at 18:30
1
$\begingroup$

Combining ARIMA and dummy indicators is a very good direction. Empirically Finding which days of the month are important is an important feature of software . See the daily demand study in http://autobox.com/cms/index.php/afs-university/intro-to-forecasting/doc_download/53-capabilities-presentation and suumarized hereenter image description here . One needs to robustly integrate these features along with any user-specified predictor series.

If you wish to post your data I will try and help further and I will introduce your data to AUTOBOX (my tool of choice which I helped to develop).

It is naive to try and separate particular days from normal days as one needs to seamlessly incorporate daily/monthly effects , level hist effects, memory effects. holiday effects without being blindsided/affected by anomalies not representing "typical behaviour" thus needing to be cleansed of the anomalous portion/effect.

EDITTED after receipt of data which include two user-suggested "helping variables" viz day1 of the month and a collective holiday indicator. The OP couldn't post his data to SE so he emailed it to me . I would be glad to forward it and any of my analytical results to any interested reader . He sent two typical time series .. One for revenue for prepaid and another for postpaid (1061 consecutive days) . The postpaid series is here enter image description here resulting in a model that provide the following actual/fit and forecast enter image description here with 90 day forecasts here enter image description here . The equation is here in two parts. enter image description here and enter image description here . There is a very strong ar(7) structure suggesting a daily memory component along with two monthly indicators.

The prepaid series was a little more interesting . Here it is enter image description here with forecasts here enter image description here . The useful model that AUTOBOX automatically provided is here in two parts enter image description here and here enter image description here . The stat summary is here enter image description here . The enter image description hereresiduals from the model are here with acf here enter image description here suggesting model sufficiency. The cleansed plot is interesting enter image description here . Finally the Actual/Fit and Forecast plot enter image description here . Note that a number of days in the month were found/discovered to be significant in addition to the user suggested variable day1.

Christmas/Year End is certainly important for both series while the prepaid series is very seasonal (monthly factors within the year) the postpaid series is also very "seasonal" with respect to days within the week.

Most ( if not all !) daily data dealing with human activity has similar characteristics as we are creatures of habit. Daily stock market price data ..... not so much as it is driven in a large part by emotion which is not as quantifiable !

Both models combined memory (arima) and deterministic structure . Note that forecasts foe future holidays were not included in the prepaid study as they were not delivered thus biasing the forecasts to the down side when holiday activity commenced.

One other comment in order to determine if individual holidays are important ( as they usually are) as no one coefficient can explain lead,contempraneous and lag structure around each holiday event , AUTOBOX needs to know the list of holidays (via resident data files) and when they have occurred and will occur in the planning horizon. This data came from Vietnam but we used a generic European/US calendar for this preliminary but highly informational study.

The fact that St. Patrick's day is important for Vietnam assuredly reflects an unspecifed holiday (coincident holiday) arriving around or on the 17th of March.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thank you very much for you help. It is very kind of you to provide so much details. However, I'm trying to approach this with R, do you have any recommendations for me to improve my model in R? What type of seasonal should I put for the coding? I'm struggling with this because the number of days each month varies from 28 to 31. Br. $\endgroup$ – Nguyen Ba Viet Apr 20 '18 at 1:13
  • $\begingroup$ I used AUTOBOX which is available R . I don't believe the current software you are using has the functionaliity that you need but perhaps you could write it. $\endgroup$ – IrishStat Apr 20 '18 at 3:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.