8
$\begingroup$

I'm exploring some Stack Overflow data. Amongst other variables, I have variables for the time questions were asked and the time they were answered.

I'm interested in predicting how long a questioner might expect to wait before their question is answered, based on the programming language.

I have 26k observations divided between 10 languages. Format-wise, I can format the time differentials between question asked and answered as integers, and group the data by programming language as neccesary.

The output of my model would ideally be very simple: just an integer/float representing the number in minutes a user might typically expect to wait before their question is answered.

What would be the most suitable modelling / machine learning technique for this (ideally in Python)? I've explored various GLM types in StatsModels but can't find something that's clearly suitable. As the data are neither continuous nor linear I don't think OLS is right. The data's obviously not binary either, so logistic regression's out. As I'm just dealing with time differentials as integers, I don't think this requires a Time Series analysis model either.

For the record this is just a personal project based on a publicly available Stack Overflow data dump. I have no affiliation with Stack Overflow.

Improve this question
$\endgroup$
2
  • 4
    $\begingroup$ Googling for "survival analysis" would be a good start. $\endgroup$
    – Tim
    Commented Oct 5, 2015 at 13:27
  • 1
    $\begingroup$ The best solutions discussed here are not from machine learning, so consider changing the title to omit machine learning. $\endgroup$
    – Frank Harrell
    Commented Oct 6, 2015 at 12:44

2 Answers 2

Reset to default
7
$\begingroup$

Waiting times can often be modelled by the exponential distribution, but this better describes the time between two events of the same type, e.g. it would probably nicely model the time between two subsequent questions.

The time to answer has some more complex properties:

  • There is a real chance of it never being answered
  • There will be some delay before an answer can realistically appear
  • Complex questions take longer (maybe approximately captured by #characters in question?)

For these reasons the exponential distribution won't be a perfect fit. The point about the delay is the most serious defect. Survival analysis might be another option, but also hasn't got the "waiting" time for people to read and understand the question, before they can answer.

Other concerns include dependence on time of day and day of week. There could be spam bots that answers the question very quickly.

Since this is exploratory anyway, I would start by plotting the data. Start with some histograms of the time until an answer, look at how many never get answered. Go the same plots by time of day and day of week. See if there are differences. Then plan from there.

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ (1) An exponential distribution can be fitted to censored time-to-event data. But, probably ruling out an exponential model more decisively than the necessary initial delay before an answer can first appear, it's clear that the hazard's far from constant over time - that a day-old question's much more likely to be answered in the next hour than a fortnight-old question. (2) Kaplan-Meir survival plots would be more suitable for exploratory analysis than histograms, allowing censored times to be included. (3) The median time to answer for each language could be easily calculated if more ... $\endgroup$
    – Scortchi ♦
    Commented Oct 5, 2015 at 15:03
  • $\begingroup$ ... than half the questions concerning that language have been answered, & would be perhaps good enough for the OP's purposes. $\endgroup$
    – Scortchi ♦
    Commented Oct 5, 2015 at 15:03
  • $\begingroup$ @Scortchi 1. I agree, the delay is the more serious defect. I should have highlighted that; I think that I did not think about the non-answered question as censoring, because I assumed that they will never be answered. 2. I was thinking about histograms with an artificial bin to the right for non-answered questions. But there is probably more than one option here. The survival plots are certainly a good option as well. 3. Maybe, but depends on how strong the time effects are. I also suspect there will be a language-time interaction. $\endgroup$
    – Erik
    Commented Oct 5, 2015 at 15:47
  • $\begingroup$ @Scortchi More on three: I think an English language question will get quicker answer at unusual local times (i.e. 01:00 a.m) since other people around the globe can and will answer in English. This might not be the case for other languages, but Spanish should get faster answers than German. $\endgroup$
    – Erik
    Commented Oct 5, 2015 at 15:49
  • $\begingroup$ (1) No, I think the non-constant hazard's likely the more serious, & obvious, defect; the delay might not greatly impact the model's fit if it's small relative to the average waiting time. It's cleaner to think of unanswered questions as having a censored time-to-answer because it spares you having to decide a cut-off time between not-answered-yet & never-will-be-answered, which will always be at least a little unsatisfactory. (2) Trouble is, differences in bin frequencies may reflect changes in the frequency of questions over time. (3) The q. refers to programming languages. $\endgroup$
    – Scortchi ♦
    Commented Oct 5, 2015 at 16:19
4
$\begingroup$

I have accepted an answer above and am only writing this as an answer because I think my research could be beneficial to other Python users.

The comments and answers above helpfully directed me towards Survival Analysis. After exploring the modelling options in StatsModels, I then discovered the library Lifelines by Cameron Davidson Pilon (@Cam.Davidson.Pilon).

I can thoroughly recommend Lifelines to anyone looking to do duration-based analysis in Python. It's excellent in many ways. I particularly like the way it addresses - both theoretically and practically - the concept of Censorship. Given how elegant and effective the library is, and how broad the applications of Survival Analysis are, I'm somewhat surprised Lifelines isn't better known.

I can also recommend watching this introductory talk. It got me up and running with Lifelines in no time.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Note that in other languages such as R, survival analysis is thoroughly known and used. The survival package in R is the gold standard. $\endgroup$
    – Frank Harrell
    Commented Oct 6, 2015 at 12:14
  • 1
    $\begingroup$ @FrankHarrell - Yes, I actually started out with R so to speak, and knew of the existence of survival analysis. I'd just never used it and thus it wasn't on my radar. But I'm very happy to have found this Python implementation. $\endgroup$
    – RDJ
    Commented Oct 6, 2015 at 12:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.