1
$\begingroup$

I am trying to predict a time-series data set, using python. I have a timestamp and number of calls in a network for this particular timestamp. I have to predict number of calls in the future. Currently, I have 90 days of data and for every 20 minutes in a day i have an entry with number of calls. enter image description here

I resample the data so i plot the mean of the data for every 3 days and i get the following results: enter image description here

enter image description here

I am not sure the trend graph is saying much. The data is going up and down so no obvious trend. However, there is seasonality. After plotting this, I checked for autocorrelation and this is where the weirdness is happening I convert the created DataFrame to series and then plot it. This results in something weird, which shows just random values and no correlation

enter image description here

I do not know, if i am doing something wrong with my data, but if i have no autocorrelation and no stationarity, should i use Time Series analysis at all? And in general can I make any predictions on this data, maybe with linear regression? I am new to data science and i am doing this for my bachelor project, so i really need help. I have read a lot on the internet and maybe at this point i am pretty confused. Any help will be appreciated!

Regards

P.S Here are some screen shots of acf and pacf plots with statsmodel library First screenshot represents the data resampled to 3 days mean: enter image description here

The acf and pacf for data resample to 1 day seems the same:

enter image description here

Here are the other plots as well for data resampled to 1 day enter image description here enter image description here

Improve this question
$\endgroup$
12
  • $\begingroup$ 5/28/18 - 9/17/18 is not 90 days . Is this 5 readings per week or 7 ? post the actual data 90x72 in a csv format (1 column) as you have 90 days (you say) and 72 readings per day ( every 20 miniutes = 3x24) and I will try and help further. $\endgroup$
    – IrishStat
    Commented Dec 1, 2018 at 17:42
  • $\begingroup$ Considering the vertical axes, is a seasonality on the order of 0.01 truly meaningful? This may be helpful. You might profit from reading an introductory forecasting testbook. I recommend the excellent free online book Forecasting: Principles and Practice (2nd ed.) by Athanasopoulos & Hyndman. $\endgroup$
    – Stephan Kolassa
    Commented Dec 1, 2018 at 17:58
  • $\begingroup$ dropbox.com/s/lbsuir54veq0ond/DatafromPython.csv?dl=0 We have more than 90 days of data - my mistake. Actually, good question, maybe this seasonality is not very meaningful. I will check the book. $\endgroup$
    – Krblaze
    Commented Dec 1, 2018 at 18:04
  • $\begingroup$ @StephanKolassa The link you send is useful regarding information, but i work with python and there is no seasonal plot likle in R there As far as I know at least $\endgroup$
    – Krblaze
    Commented Dec 1, 2018 at 18:11
  • $\begingroup$ whys is your data all 1's ? $\endgroup$
    – IrishStat
    Commented Dec 1, 2018 at 18:21

2 Answers 2

Reset to default
0
$\begingroup$

You ask .... what to do ...

I say I took your 121 daily values into AUTOBOX whose promary objective is to assess predictability from a sequence of observations for an interesting series to forecast possibly using ( not in this case ) user suggsted predictors . enter image description here

Your series is a discrete series insofar as only a partictular set of values can be observed (71,72,73,74,,,)

AUTOBOX looks for predictability using prior values (ARIMA) and in this case daily effects and possible changes in daily effects.

The equation ( with identified features ) is here enter image description here , The suggestion is that the only identified feature was a change in day6 at period 83 (of 121) suggesting that week 1-11 was different from week 12-17 .

This suggests forecasts here enter image description here

Overall the Actual/Fit and Forecast graph is here enter image description here

The confidence intervals around the forecast are asymmetrical and include the possibility of future anomalies.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Okay, what i see from the graphs is that we cannot make accurate forecasting in my case? the last graph shows some forecasting but i cannot see how it fits with actual data. I am sorry but i have not worked with AUTOBOX. What i got is that i just have one change between week 1-11 and week 12-17. My primary question was: "...if i have no autocorrelation and no stationarity, should i use Time Series analysis at all? " Mainly focusing on the statement that i do not have autocorrelation thank you for the time you spend! :) $\endgroup$
    – Krblaze
    Commented Dec 7, 2018 at 0:25
  • $\begingroup$ Just because there is no autocorrelation ..it does not mean that there are lo latent determinstic structure like a seaonal pulse for day 6. Look at the following $\endgroup$
    – IrishStat
    Commented Dec 7, 2018 at 2:53
  • $\begingroup$ group 1 73,74,72,72,72,73,73,73,72,72,73; group 2 76 75 75 75 74 76 73 $\endgroup$
    – IrishStat
    Commented Dec 7, 2018 at 2:54
  • $\begingroup$ see how day 6 is significantly higher in the send group $\endgroup$
    – IrishStat
    Commented Dec 7, 2018 at 2:58
  • $\begingroup$ Okay i got it now :)) i will try to use ARIMA and explore my data better $\endgroup$
    – Krblaze
    Commented Dec 7, 2018 at 11:31
1
$\begingroup$

Pandas isn't that well suited to analyze autocorrelation, that might be a source of problems in your data. The Statsmodels library in Python has better options.

Try: statsmodels.tsa.stattools.acf and statsmodels.tsa.stattools.pacf

You're also looking at the mean of every 3 days of data, when it is likely that your data has 24 hour seasonality and 7 days seasonality, and both will get muddied by averaging over 3 days.

Improve this answer
$\endgroup$
9
  • $\begingroup$ Thanks for the suggestion. I did plots with statsmodels for acf and pacf. I got some results that did not change much. I have added some screenshots in the original post. So as you can see still no autocorrelation. What would you suggest - should I use linear regression or still time series but change something? $\endgroup$
    – Krblaze
    Commented Dec 3, 2018 at 21:06
  • $\begingroup$ @Kosi: If interested in predicting future number of calls, will you want to predict the number of calls in the next hour (or next few hous), next day (or next few days), next week (or next few weeks), etc.? That should determine the level of temporal aggregation for your time series. For instance, if interested in predicting the number of calls in the next day (or next few days), it will be convenient to work with a time series of daily numbers of calls. $\endgroup$
    – Isabella Ghement
    Commented Dec 6, 2018 at 22:04
  • $\begingroup$ If you have both trend and seasonality present in your (aggregated) time series, your forecasting model will have to reflect that. For forecasting purposes, you could use a time series regression model which includes trend, seasonality and possibly autocorrelated errors. Or you could use a seasonal ARIMA model. In any event, the ACF and PACF plots of the (aggregated) time series will respond to the presence of trend and seasonality in this series, so you should expect upfront to see some strong features in these plots. $\endgroup$
    – Isabella Ghement
    Commented Dec 6, 2018 at 22:08
  • $\begingroup$ The real questions are: 1. What is the appropriate level of temporal aggregation for your original time series (if any)? 2. Given this level of aggregation, does the aggregated time series display evidence of trend and/or seasonality? 3. What kind of forecasting model will you use and how are you going to capture trend and/or seasonality in your model? (Once you settle on a model, you can refine it to allow for further nuances present in your data - but you have to start somewhere.) 4. How will you evaluate the suitability of your forecasting model for the data & the forecast accuracy? $\endgroup$
    – Isabella Ghement
    Commented Dec 6, 2018 at 22:12
  • $\begingroup$ @Kosi: Are you aggregating your data (e.g., computing total number of calls for each day) or merely changing the sampling frequency? In any event, your decomposition plots show a locally changing trend over time as well as seasonality, so I am not sure why you would believe neither of these features exist in your data. $\endgroup$
    – Isabella Ghement
    Commented Dec 7, 2018 at 0:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.