6
$\begingroup$

(I apologise for being a newb, but I'm a researcher introducing myself to data mining---any help or insight would be greatly appreciated. Also, this isn't technically a homework question, but I've marked it as such to distinguish that this is a broad question from a non-specialist).

I'm having trouble finding the correct approach to my (fairly simple) example.

Let's say I have months of data for log-in times of a certain website. The data has been selected and cleaned such that I have a list of Date_Time for each log-in.

Now, suppose I wanted to predict the log-ins for the next two weeks by day and hour, based on these past trends.

I imagine I would cluster the data by day (assuming beforehand that there will be different trends with respect to Monday vs. Friday) and make some regression analysis to predict the next two (say) Mondays.

Similarly, I could cluster by the hour and do a regression analysis to extrapolate the trend of log-ins.

Anyone know of a resource which tells you how to do this in Python? I want to keep this example fairly straightforward, but I'm open to any more ideas on how to model this behavior more efficiently.

Improve this question
$\endgroup$
7
  • 3
    $\begingroup$ This is time series data . You should first consider something like a seasonal ARIMA model rather than do regression on time. The differencing and seasonal differencing can handle the trends. This does not sound like a data mining problem. It sounds like a standard time series problem. if you have covariates that are fixed or time varying i am sure you can incorporate that in the model. The problem with fitting simple trends and seasonal terms like a sine or cosine of time is that there may still be autocorrelation in the residual series, something ordinary regression doe not account for. $\endgroup$
    – Michael R. Chernick
    Commented Jul 20, 2012 at 18:10
  • $\begingroup$ Trying an ARIMA model sounds like a great idea, though I'm not sure in this example whether taking into account seasonal variation is necessary. Considering the data I have, it's more reasonable to just forecast the future log-ins by weekday. A simple trend would do. Do you have any recommedations for learning how to implement what you describe above? $\endgroup$
    – eherrtelle59
    Commented Jul 20, 2012 at 19:03
  • $\begingroup$ The models are old but Box and Jenkins formalized the ARIMA model building methodology in the late 1960s culminating in their classic book in 1970. Most modern time series books that work in the time domain cover this. I would recommend the most recent edition of the Box and Jenkins book by Box, Jenkins and Reinsel. Jenkins is diseased so Box and Reinsel did this latest update. Here is an amazon link to it: $\endgroup$
    – Michael R. Chernick
    Commented Jul 20, 2012 at 19:09
  • 1
    $\begingroup$ amazon.com/Time-Analysis-Forecasting-Probability-Statistics/dp/… $\endgroup$
    – Michael R. Chernick
    Commented Jul 20, 2012 at 19:20
  • $\begingroup$ @ehentelle In our community IrishStat is a man who has made a company and a career out of automating and generalizing the Box Jenkins approach to time series analysis with his software package autobox. He will I am sure be happy to demonstrate his software on your data and provide you with a good solution with this general methodology. $\endgroup$
    – Michael R. Chernick
    Commented Jul 20, 2012 at 19:24

1 Answer 1

Reset to default
1
$\begingroup$

I am in agreement with the commentators, this is a simple time-series problem. If you are unconcerned with seasonal changes, I'm not sure what you expect to get out of simple day and hourly counts.

ARIMA is what you want. If you really need something that is specifically machine learningish, just try basic Bayesian modeling. It would incorporate your prior data and it is the basis of most ML paradigms....

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.