Since My data frequency is daily but data is not available on Weekends( Sat and Sun) and Holidays.Therefore nearly 30-40 % data is missing with sequential time series. So I don't know how to deal with Non sequential time series data.
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
Impute values for missing holiday values (or any values) by taking an average of the day before and the day after. This is rudimentary just to get going and can ultimately be modified/corrected via Intervention Detection procedures. Set your seasonality to 5 and jointly identify arima and latent deterministic effects like daily effects , pulses , step-level shifts , seasonal pulses i.e. changes in daily effects over time and local time trends following http://docplayer.net/12080848-Outliers-level-shifts-and-variance-changes-in-time-series.html and an iterative modelling process similar to https://autobox.com/pdfs/ARIMA%20FLOW%20CHART.pdf . Follow this thread for a practical example Simple method of forecasting number of guests given current and historical data .
You also might load up 11 monthly indicators to potentially capture monthly effects (and/or weekly dummies to capture possible weekly effects ) as you form your SARMAX model as here suggested here https://autobox.com/pdfs/SARMAX.pdf
Daily data usually has anthropogenic effects possibly reflecting systematic activity/behavior requiring inclusion of possible empirically developed deterministic factors. ARIMA memory is a poor substitute and widely over-used when this is the case. Only your data knows for sure as to how to form your model .. sometimes with domain knowledge in place.
Fitting a guessed model with a pre-specified # of fourier/trigometric terms and assumed forms of trends without detecting level shifts or changes in trends or changes in daily effects or changes in error variance should be studiously avoided whenever possible as it is not model identification but model specification/ guessing/presumption. Models like these do not lead to any reliable tests of significance or forecast error variance allowing for future anomalies to be considered.
$\begingroup$
$\endgroup$
I don't think this should be a problem in principle. It depends on what's your goal and the type of data you have.
Maybe you can have good results treating the data as sequential (i.e. just dropping the missing points) or using some imputation method for the weekends. Or you can exploit a trend-seasonal decomposition for your prediction: e.g. prediction for next Monday will be the (trend adjusted?) average of all Mondays in the past 3 months.
$\begingroup$
$\endgroup$
The FASSTER model (https://github.com/tidyverts/fasster) is capable of modelling data available only on particular days as you describe. Fill in missing data for weekends and holidays with zero, and then model those days with no terms in the model (giving zero forecasts).
To do this, you could use the conditional modelling states with %?%. The syntax for the model would be:
fasster(response ~ condition %?% (your_model))
On the left of %?% is a variable or function which returns either TRUE or FALSE. Only when it is TRUE, the states specified in your_model will be included in the model. The best way to use this would be to write a small function which determines if it is a day with data or not, as future values of the index are generated automatically by forecast(). It is not a problem if data is used, but future values will have to be specified via new_data (much like exogenous regressors).
Here's an example of how this can done for detecting weekdays (which can be extended to also detect holidays):
is_weekday <- function(time){
lubridate::wday(time) %in% 2:6 # Monday - Friday
}
and to use this function in the model formula:
fasster(response ~ is_weekday(your_index_variable) %?% (your_model))
There are some examples in the README linked above for what is allowable in your_model.
answered Feb 26, 2020 at 14:18