6
$\begingroup$

I am building a user login prediction system. This is my first time building any prediction system. Main aim is to predict when a user might login next in future. That is i need to predict "Time". I have a dataset of hundreds of users and when did they login. It is shown below. I only have login time and nothing else. no activity time or logout time. i have gigabytes of such data. 

+----+----------------------------+----------------------------+----------+-----------------+
| Id | Time_Stamp                 | Login_Time                 | Day      | Account_name    |
+----+----------------------------+----------------------------+----------+-----------------+
|  1 | 2019-01-31 00:01:35.818000 | 00:01:35.818000         | Thursday | user1           |
|  2 | 2019-01-31 00:01:35.885000 | 00:01:35.885000         | Thursday | user2          |
|  3 | 2019-01-31 00:01:26.335000 | 00:01:26.335000         | Thursday | user2          |
|  4 | 2019-01-31 00:01:35.885000 | 00:01:35.885000         | Thursday | user1          |
|  5 | 2019-01-31 00:01:26.336000 | 00:01:26.336000         | Thursday | user2          |
|  6 | 2019-01-31 00:01:29.842000 | 00:01:29.842000         | Thursday | user3       |
|  7 | 2019-01-31 00:01:35.819000 | 00:01:35.819000         | Thursday | user4           |
|  8 | 2019-01-31 00:01:26.336000 | 00:01:26.336000         | Thursday | user7          |
|  9 | 2019-01-31 00:01:35.821000 | 00:01:35.821000         | Thursday | user5       |
| 10 | 2019-01-31 00:01:35.886000 | 00:01:35.886000         | Thursday | user4          |
+----+----------------------------+----------------------------+----------+-----------------+

My approach

1) Store data into data base and get data for each user as the original dataset has data combined for all users. 2) Run machine learning algorithms on each user and predict when he will login next 3) cross verify predicted result with actual result when the user does login.

Machine learning algorithm I am thinking is linear regression or SVM but the issue is in some user the data is too less and some is too much. so I need some algorithm which will work for both. Also I am planning to combine pattern recognition and time series prediction algorithms to get more accuracy.

Another issue is what variable should i take into account. Should I just pass login date and login time of each user and let the algorithm predict next? or is there any better way? i need suggestion whether my approach is correct or not? How can I predict time? Do libraries support that or do I need to write my own algorithm?

Most Machine learning problems take some sort of numbers into consideration like stock market prediction but how should i take "time" into consideration?

Improve this question
$\endgroup$
10
  • 1
    $\begingroup$ It seems like you are leaving a lot of data on the table that could be useful to you. I'd imagine you'd have some basic demographics of users that could be powerful predictors. For example, time-zone, age, gender, etc. $\endgroup$
    – StatsStudent
    Commented Jan 31, 2019 at 22:58
  • $\begingroup$ @StatsStudent No. thats the biggest issue. I only have what is shown in the table above. $\endgroup$
    – Sam
    Commented Jan 31, 2019 at 22:59
  • 2
    $\begingroup$ OK. Then it looks like you'll need to use just the limited number of variables that you have. You can try running the algorithms separately for each user or your model might perform better by running the algorithm on the entire dataset and then making individual predictions (for example by treating User as a random effect in you use a regression modelling approach). Regardless it would be imperative to split your data into training and validation datasets, being sure to select the algorithm that leads to the smallest prediction error. $\endgroup$
    – StatsStudent
    Commented Jan 31, 2019 at 23:09
  • 1
    $\begingroup$ If you're privy to network traffic statistics or load, that may be helpful too. $\endgroup$
    – StatsStudent
    Commented Jan 31, 2019 at 23:12
  • 1
    $\begingroup$ One good place to start looking may be churn analysis. Churn analysis deals with why people stop using a service, and usage patterns (including login patterns) are a key part of that. Some of the techniques used in such analyses may be of use to you. r-bloggers.com/churn-analysis-part-1-model-selection towardsdatascience.com/… $\endgroup$
    – Ingolifs
    Commented Jan 31, 2019 at 23:53

1 Answer 1

Reset to default
4
$\begingroup$

Without having seen the data, I don't think machine learning is the appropriate choice here because your data is not IID.

It is reasonable to believe that users may have different login habits, and we should allow for that in our model by modelling time between logins as a hierarchical model.

Posit that each user's time between logins has some distribution (maybe it is exponential), which is parameterized by $\lambda_i$. Then, maybe you say that the $\lambda_i$ come from some distribution.

So your model posits the following:

$$ (\Delta_{t})_{i} \sim \operatorname{Exponential}(\lambda _i) \quad i = 1 \dots N $$

$$ \lambda_i \sim P(\lambda)$$

Here $P$ is the distribution for the $\lambda_i$. Maybe the $\lambda$ are gamma distributed or something. I'm not saying that this is the model, but I think an approach like this is reasonable. If you have a lot of data, you could even do an Empirical Bayes approach and construct "priors" from the data.

Improve this answer
$\endgroup$
4
  • $\begingroup$ (+1) Seems reasonable to me. $\endgroup$
    – StatsStudent
    Commented Feb 3, 2019 at 17:24
  • $\begingroup$ very sorry for late response. your answer seems reasonable to me as well. my approach is quite similar to yours. $\endgroup$
    – Sam
    Commented Feb 6, 2019 at 17:42
  • $\begingroup$ Also can you suggest some ways to take time into consideration? should I separate time into hours, minutes and seconds then pass them into library or is there any other eay? $\endgroup$
    – Sam
    Commented Feb 6, 2019 at 18:21
  • $\begingroup$ You can convert the time to fractional minutes, or days if the login times are sufficiently long. $\endgroup$
    – Demetri Pananos
    Commented Feb 6, 2019 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.