Suggestions on Modeling Approach to Model Percent Complete of a Task

Question

I am trying to predict what percentage (or proportion) of a task is completed by various workers, given the time left until the deadline to complete the task and I'm looking for help on how to approach modeling this.

I have historic data which contains "worker ids" (WorkerID) that uniquely identify each worker, the number of days left to complete the task or DaysToDeadline (e.g. 25, 24, 23, etc.), and the Percentage of work completed at the given number of days to deadline (PercentComplete).

Generally speaking the percentage completed will always increase, but can sometimes revert to smaller percentage completed, if for example, the worker makes a mistake during the task and has to redo previously completed work. If a worker completes a task early, he can begin work on another task, so his "percent completed" can actually go above 100% and is recorded as such. In addition, there is not necessarily an equal number of data points for each worker since some workers could start on the task earlier or later than others.

My sample data looks like this:

WorkerID    DaysToDeadline  PercentComplete
1   25  0
1   24  2
1   23  2
1   22  5
1   21  10
2   25  5
2   24  6
2   23  7
2   22  10
2   21  7
2   20  10
3   25  0
3   24  5
3   23  0
4   25  10
4   24  20
4   23  25
4   22  26
4   21  30
4   20  50
4   19  66
4   18  80
4   17  96
4   16  100
4   15  106

Since I need to make individual level predictions and obtain confidence intervals for these predictions, I was thinking about possibly using some sort of generalized linear mixed model where I treat worker ID as a random effect, Days to deadline and percent complete as fixed effects. I thought about using a logistic or beta family model, but since I get get things like 105%, I don't think this would be appropriate. So, I'm looking for some suggestions how how to possibly approach this? I'm ideally looking for a regression approach, but would be open to others such as machine learning approaches too -- I'm just more familiar with the regression approach. Thanks.

UPDATE:

If it's too difficult to suggest a modeling approach to this problem due to the fact that the percentages can exceed 100 (e.g. 105%), I'd be amendable to simply truncating or modifying the definition of the task completion percentage so that 100% is the highest percentage complete that would be possible.

Answer 1

I believe you would be best served by transforming the data to have something like "amount of work done" and "time spent working" and then predict "amount of work done per time unit" - this would mostly get rid of the problems with <0 and > 100 percent complete.

Obviously we can use "percent complete" to estimate the amount of work done, but that won't be without problems (especially if tasks differ). In this case, we can transform the data by subtracting the measured time steps to get something like:

WorkerID DayDiff CompleteDiff PercentStarted
1  1  2  0
1  1  0  2
1  1  3  2
...
2  1  1  5
2  1  1  6
...

DayDiff may be unnecessary if you have no gaps in your data. Now you could use a hierarchical model to predict WorkPerDay = CompleteDiff / DayDiff. What should be the response distribution? This is hard to say without looking at the complete dataset, but it is possible normal wouldn't be terrible. I would however guess it would be better fit by a mixture - one component to model the negative steps and one to model the positive steps, as I would assume negative steps occur for very different reason than positive steps. If your data are discrete, it is possible something like negative binomial would work good for either component.

The neat part about using a mixture model is that the mixture parameter itself would be of interest (how likely is the worker to have negative progress).

If you cannot easily fit mixtures, it could make sense to fit a separate model to predict a binary outcome "positive" vs. "negative" progress and then separately regress on the positive and negative outcomes.

In any case, you should check the predictions of your model against the actual dataset (known as posterior predictive checks in the Bayesian world, but works also in the frequentist case) to see if your model fits. E.g. what is the maximum progress your model predicts compared to maximum in the data, does it predict similar number of zeroes as are in the data etc. If those checks are way off you are likely to be using a wrong response distribution.

Answer 2

This problem is analogous to survival analysis.

For each observation, the model predicts the length of time before an event happens. This is usually death in biological contexts or mechanical failure in engineering contexts. In your case, this would the number of days before the project is complete. Instead of predicting survival time, you want to predict time to completion.

Each worker is one observation. Each projects have a different number of days before the deadline, or "lifetime". If a project is less than finished by the deadline, then the project lifetime is said to be "right censored". In the biology context, this would mean that the subject died after the observation period.

This would answer the question "how many projects will go unfinished on the day of the deadline?"

Here is an example of the output from a survival analysis. Just replace "number alive" with the number of unfinished projects and "number of deaths" with the number of finished projects.

There is already vast literature on this topic. One off-the-shelf choice is a random survival forest which also has an R implementation in the package randomForestSRC.