3
$\begingroup$

I am trying to predict what percentage (or proportion) of a task is completed by various workers, given the time left until the deadline to complete the task and I'm looking for help on how to approach modeling this.

I have historic data which contains "worker ids" (WorkerID) that uniquely identify each worker, the number of days left to complete the task or DaysToDeadline (e.g. 25, 24, 23, etc.), and the Percentage of work completed at the given number of days to deadline (PercentComplete).

Generally speaking the percentage completed will always increase, but can sometimes revert to smaller percentage completed, if for example, the worker makes a mistake during the task and has to redo previously completed work. If a worker completes a task early, he can begin work on another task, so his "percent completed" can actually go above 100% and is recorded as such. In addition, there is not necessarily an equal number of data points for each worker since some workers could start on the task earlier or later than others.

My sample data looks like this:

WorkerID    DaysToDeadline  PercentComplete
1   25  0
1   24  2
1   23  2
1   22  5
1   21  10
2   25  5
2   24  6
2   23  7
2   22  10
2   21  7
2   20  10
3   25  0
3   24  5
3   23  0
4   25  10
4   24  20
4   23  25
4   22  26
4   21  30
4   20  50
4   19  66
4   18  80
4   17  96
4   16  100
4   15  106

Since I need to make individual level predictions and obtain confidence intervals for these predictions, I was thinking about possibly using some sort of generalized linear mixed model where I treat worker ID as a random effect, Days to deadline and percent complete as fixed effects. I thought about using a logistic or beta family model, but since I get get things like 105%, I don't think this would be appropriate. So, I'm looking for some suggestions how how to possibly approach this? I'm ideally looking for a regression approach, but would be open to others such as machine learning approaches too -- I'm just more familiar with the regression approach. Thanks.

UPDATE:

If it's too difficult to suggest a modeling approach to this problem due to the fact that the percentages can exceed 100 (e.g. 105%), I'd be amendable to simply truncating or modifying the definition of the task completion percentage so that 100% is the highest percentage complete that would be possible.

$\endgroup$
  • 1
    $\begingroup$ Do you have task IDs too ? Do the same workers work on completely different and independent tasks or is there overlap ? The issue of >100% completion seems very strange: in your sample data, why doesn't the last row show worker 4 starting a new task with 6% complete ? $\endgroup$ – Robert Long May 16 at 11:21
  • $\begingroup$ is task dificulty the same for every percantage? I.e. to go from 5% to 10% is as hard as to go from 75% to 80%? $\endgroup$ – rep_ho May 16 at 13:44
  • $\begingroup$ Hi, @RobertLong. Good question. Yes, I have different task IDs, but to be honest, for this particular analysis, I think it's appropriate to treat everything as the same exact task since the tasks are essentially identical. Further down the road when I plan to move on to more complicated analyses across tasks, I might come back to this, but for the time being, it's sufficient to assume that everyone is working on the same task. $\endgroup$ – StatCurious May 16 at 16:46
  • $\begingroup$ @rep_ho, I'm not really sure. It might be that initially the first half of the task is done very quickly (so the movement from zero to 50% happens quite quickly), but that the progression from 50% to 100% is much slower. I'm honestly, just not sure. I suppose I'd be happy to explore models that make either assumption. Perhaps if it's more generalizable to simply assume that the differential in completion percentage between days is not identical (or nearly so), I should stick to that assumption. I suppose it's also possible some workers are consistent and some are not from day to day. $\endgroup$ – StatCurious May 16 at 16:51
  • 1
    $\begingroup$ Not really. I need much more detail. For one thing, what exactly does each row represent ? Even though you don't want want to include taskID it would help my understanding if you included all relevant data. $\endgroup$ – Robert Long May 16 at 17:54
0
$\begingroup$

I believe you would be best served by transforming the data to have something like "amount of work done" and "time spent working" and then predict "amount of work done per time unit" - this would mostly get rid of the problems with <0 and > 100 percent complete.

Obviously we can use "percent complete" to estimate the amount of work done, but that won't be without problems (especially if tasks differ). In this case, we can transform the data by subtracting the measured time steps to get something like:

WorkerID DayDiff CompleteDiff PercentStarted
1  1  2  0
1  1  0  2
1  1  3  2
...
2  1  1  5
2  1  1  6
...

DayDiff may be unnecessary if you have no gaps in your data. Now you could use a hierarchical model to predict WorkPerDay = CompleteDiff / DayDiff. What should be the response distribution? This is hard to say without looking at the complete dataset, but it is possible normal wouldn't be terrible. I would however guess it would be better fit by a mixture - one component to model the negative steps and one to model the positive steps, as I would assume negative steps occur for very different reason than positive steps. If your data are discrete, it is possible something like negative binomial would work good for either component.

The neat part about using a mixture model is that the mixture parameter itself would be of interest (how likely is the worker to have negative progress).

If you cannot easily fit mixtures, it could make sense to fit a separate model to predict a binary outcome "positive" vs. "negative" progress and then separately regress on the positive and negative outcomes.

In any case, you should check the predictions of your model against the actual dataset (known as posterior predictive checks in the Bayesian world, but works also in the frequentist case) to see if your model fits. E.g. what is the maximum progress your model predicts compared to maximum in the data, does it predict similar number of zeroes as are in the data etc. If those checks are way off you are likely to be using a wrong response distribution.

$\endgroup$
0
$\begingroup$

This problem is analogous to survival analysis.

For each observation, the model predicts the length of time before an event happens. This is usually death in biological contexts or mechanical failure in engineering contexts. In your case, this would the number of days before the project is complete. Instead of predicting survival time, you want to predict time to completion.

Each worker is one observation. Each projects have a different number of days before the deadline, or "lifetime". If a project is less than finished by the deadline, then the project lifetime is said to be "right censored". In the biology context, this would mean that the subject died after the observation period.

This would answer the question "how many projects will go unfinished on the day of the deadline?"

Here is an example of the output from a survival analysis. Just replace "number alive" with the number of unfinished projects and "number of deaths" with the number of finished projects.

enter image description here

There is already vast literature on this topic. One off-the-shelf choice is a random survival forest which also has an R implementation in the package randomForestSRC.

New contributor
Sam Castillo is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.