What ML algorithm(s) should I learn in order to predict based on multiple features the likelihood of a binary event occurring in the next week?

What ML algorithms should I learn in order to predict the likelihood of an event occurring in a specified time period from the present forward based on multiple features in historical data I've collected, including past occurrences of the event I'm trying to predict?

I'm trying to predict the likelihood of a binary event occurring in the next week based on ~10 features to begin with, though that number may grow in the future. If I would need to switch approaches in the future as my number of features grow, I'd appreciate knowing an approximate number of features at which point I should start thinking about switching.

Eventually this system will be a part of a website in which I'll need to be able to query this stat on multiple individuals, multiple times a day, based on each individual's changing behavior. The high-level purpose is to predict the likelihood of a user having a setback on a website designed to help people break bad habits.

• Are you comfortable with logistic regression? Do you already use some algorithms? Do you know what an ARMA model is? – usεr11852 Dec 16 '15 at 19:46
• @usεr11852 No, no, and no. I've taken part of a course on basic machine learning and gradient descent, but I'm pretty much a newbie--hence, the request for direction. – Nathan Arthur Dec 16 '15 at 20:04
• Cool. I needed to clarify these points before answering. – usεr11852 Dec 16 '15 at 21:08

In addition to Marc Claesen's good advice (+1).

The logistic regression mentioned is the "most basic" binary classification algorithm. I would definitely recommend looking it up before anything else so you get an idea of what you are dealing with. You need to fit an generalized linear model - look at the glm function in R. You will be able to interpreter your results easily. To that extent, using a linear kernel will get you very similar results with logistic regression. In fairness, generally, non-linear kernels seem to be better than linear kernels for classification tasks but this is not absolute and SVMs are more complex to interpreter than logistic regressions.

Having said all this I think that both approaches mentioned (SVM and LR) throw a tonne of information out of the window for your particular task. You have time-series. Not only that, you have a binary time-series. Not only that you have strong a auto-regressive component (I can't think of the times I identify a bad habit, broke it for a bit and then started it over again). And your data are extremely likely to have subject specific effects. In other words you have a GLMM with autoregressive random effects. You can fit these guys using R's MCMCglmm.

Honourable mentions:

1. "binary timeseries prediction"; ARMA models come into play there (that's why I asked). In your case, a GLMM with autoregressive error structure should give similar results with a ARMA but I do not know enough about binary ARMAs to send you to a particular implementation.
2. "Hidden Markov Models"; HMMs essentially model transition probabilities by identifying patterns. And really this is what you want to do. They are a lot reference on the matter; usually the relate to NLP (predict the next utterance) but you solve the same problem with them in the end of the day.

So to recap, try a simple GLM first to do a logistic regression. Afterwards, you might want to amp your game by using kernel generated features (so go with Marc's suggestion with the SVM) or by directly exploit the longitudinal and subject-specific nature of your data (so check the GLMM). These two should probably give you almost everything you can get out of your data with a first pass.

• My goodness, I can see I have a lot to learn. Thank you; this gives me quite a bit to go on! I'll start looking into GLMM's. – Nathan Arthur Dec 16 '15 at 21:52
• I am glad I could help. Marc's suggestion is also quite good and I suspect that as a newcomer it will get you good mileage somewhat faster than a GLMM (I think). GLMMs will be somewhat harder to setup quickly but I think that they are more "natural" for your final task. – usεr11852 Dec 16 '15 at 22:00

Any binary classification approach will do. In this category support vector machines usually perform quite well, even without a lot of data. Given the low input dimensionality (10), you can likely benefit from nonlinear kernel functions, such as the RBF kernel.

An important step in machine learning is optimizing hyperparameters ($C$ and $\gamma$ for an SVM with RBF kernel). For that task you could use software like Optunity.

You wouldn't need to change algorithm unless you plan to get a significant amount of additional features or huge amounts of data.

• Thank you! This definitely gives me some things to start looking into. I'm guessing I'll be trying to wrap a Python library to do this, unless I stumble across a PHP solution that's close to good to go, which I doubt from what I've read so far. – Nathan Arthur Dec 16 '15 at 21:55
• Python is the weapon of choice for these things. I recommend using scikit-learn for learning algorithms and data handling and Optunity for hyperparameter optimization. – Marc Claesen Dec 17 '15 at 7:49