9
$\begingroup$

I don't know much about statistics so bear with me. Let's say I have a set of 1000 workers. I want to figure out who the hardest worker is but I can only measure the amount of work getting done in groups of 1-100 over an hours worth of work. Assuming each worker always does around the same amount of work, over a large number of trials and combinations can I rank my workers by who works hardest?

Note: this is just a metaphor, so don't worry about actually running the tests, just assume I already have a large set of data.

Edit: When I say "Assuming each worker always does around the same amount of work" I mean each individual does around the same amount of work on a day to day basis. So Joey will do around 100 units of work each day and Greg will do around 50. The problem is I can only observe the number of units of work done by the group.

More edits: In regards to the number of workers working at once and the frequency of them working. There could be any number of workers working at the same time. Some workers will probably end up working a lot more than others, that is to say, we can assume some workers will be working almost 90% of the time and others almost never.

I know that makes it difficult but I will have a very large dataset so hopefully that makes it a little bit easier.

For each hour we know which workers are working and how much work got done. From that information I want to find out who is doing the most work.

If the data were in JSON format it would look something like this:

[
  {
    "work_done": 12345,
    "Workers": [ "andy", "bob", "cameron", "david" ]
  },
  {
    "work_done": 432,
    "Workers": [ "steve", "joe", "andy"]
  },
  {
    "work_done": 59042,
    "Workers": [ "bob", "aaron", "michelle", "scott", "henry" ]
  },
  ...
]
$\endgroup$
  • 3
    $\begingroup$ Is work additive, such as the amount of some product? Then you can use linear regression to estimate the contribution from each worker, and sort workers by their coefficients. If it isn't additive then you might want something more complicated. $\endgroup$ – Douglas Zare Oct 4 '12 at 21:12
  • $\begingroup$ If you assume you know how much work each group has done and also assume that the work is evenly distributed among the participants in each group, you can simply divide the amount of work done by the group by the number of people in it and the sum up the bits of work each worker has done in different groups. This doesn't really have anything with statistics, though. $\endgroup$ – Qnan Oct 4 '12 at 21:29
  • 1
    $\begingroup$ @DouglasZare Yes, the work is additive $\endgroup$ – Greg Guida Oct 4 '12 at 21:38
  • 1
    $\begingroup$ I think the description is clear. You only observe the workers in groups and want to make inference about the individual workers. For example, say you have a total of 5 workers, on day one you observe workers {1,2,3} together, on day two you see workers {1,4,5}, on day three {2,3,4}, etc. and your data is the total output on each day. Then, can you estimate the mean output of each individual worker? The answer is yes - if you can derive the distribution of the sum of the workers then you could write down the likelihood and maximize as a function of the individual means. $\endgroup$ – Macro Oct 4 '12 at 23:31
  • 1
    $\begingroup$ What am I missing? I still fon't see where you get the individual means even. Do we always know which workers are in aparticular hour of data? Is the total amount of work per hour somehow fixed? Is there an assumption that is clear in the problem definition that I am missing? $\endgroup$ – Michael R. Chernick Oct 5 '12 at 3:14
10
$\begingroup$

David Harris has provided a great answer, but since the question continues to be edited, perhaps it would help to see the details of his solution. Highlights of the following analysis are:

  • Weighted least squares is probably more appropriate than ordinary least squares.

  • Because the estimates may reflect variation in productivity beyond any individual's control, be cautious about using them to evaluate individual workers.


To carry this out, let's create some realistic data using specified formulas so we can evaluate the accuracy of the solution. This is done with R:

set.seed(17)
n.names <- 1000
groupSize <- 3.5
n.cases <- 5 * n.names  # Should exceed n.names
cv <- 0.10              # Must be 0 or greater
groupSize <- 3.5        # Must be greater than 0
proficiency <- round(rgamma(n.names, 20, scale=5)); hist(proficiency)

In these initial steps, we:

  • Set a seed for the random number generator so anybody can exactly reproduce the results.

  • Specify how many workers there are with n.names.

  • Stipulate the expected number of workers per group with groupSize.

  • Specify how many cases (observations) are available with n.cases. (Later a few of these will be eliminated because they correspond, as it happens at random, to none of the workers in our synthetic workforce.)

  • Arrange for the amounts of work to differ randomly from what would be predicted based on the sum of each group's work "proficiencies." The value of cv is a typical proportional variation; E.g., the $0.10$ given here corresponds to a typical 10% variation (which could range beyond 30% in a few cases).

  • Create a workforce of people with varying work proficiencies. The parameters given here for computing proficiency create a range of over 4:1 between the best and worst workers (which in my experience may even be a bit narrow for technology and professional jobs, but perhaps is wide for routine manufacturing jobs).

With this synthetic workforce in hand, let's simulate their work. This amounts to creating a group of each workers (schedule) for each observation (eliminating any observations in which no workers at all were involved), summing the proficiencies of the workers in each group, and multiplying that sum by a random value (averaging exactly $1$) to reflect the variations that will inevitably occur. (If there were no variation at all, we would refer this question to the Mathematics site, where respondents could point out this problem is just a set of simultaneous linear equations which could be solved exactly for the proficiencies.)

schedule <- matrix(rbinom(n.cases * n.names, 1, groupSize/n.names), nrow=n.cases)
schedule <- schedule[apply(schedule, 1, sum) > 0, ]
work <- round(schedule %*% proficiency * exp(rnorm(dim(schedule)[1], -cv^2/2, cv)))
hist(work)

I have found it's convenient to put all the workgroup data into a single data frame for analysis but to keep the work values separate:

data <- data.frame(schedule)

This is where we would begin with real data: we would have the worker grouping encoded by data (or schedule) and the observed work outputs in the work array.

Unfortunately, if some workers are always paired, R's lm procedure simply fails with an error. We should check first for such pairings. One way is to find perfectly correlated workers in the schedule:

correlations <- cor(data)
outer(names(data), names(data), paste)[which(upper.tri(correlations) & 
                                             correlations >= 0.999999)]

The output will list pairs of always-paired workers: this can be used to combine these workers into groups, because at least we can estimate the productivity of each group, if not the individuals within it. We hope it just spits out character(0). Let's presume it does.

One subtle point, implicit in the foregoing explanation, is that the variation in the work performed is multiplicative, not additive. This is realistic: the variation in output of a large group of workers will, on an absolute scale, be greater than the variation in smaller groups. Accordingly, we will get better estimates by using weighted least squares rather than ordinary least squares. The best weights to use in this particular model are the reciprocals of the work amounts. (In the event some work amounts are zero, I fudge this by adding a small amount to avoid dividing by zero.)

fit <- lm(work ~ . + 0, data=data, weights=1/(max(work)/10^3+work))
fit.sum <- summary(fit)

This should take just one or two seconds.

Before going on we ought to perform some diagnostic tests of the fit. Although discussing those would take us too far afield here, one R command to produce useful diagnostics is

plot(fit)

(This will take a few seconds: it's a large dataset!)

Although these few lines of code do all the work, and spit out estimated proficiencies for each worker, we wouldn't want to scan through all 1000 lines of output--at least not right away. Let's use graphics to display the results.

fit.coef <- coef(fit.sum)
results <- cbind(fit.coef[, c("Estimate", "Std. Error")], 
             Actual=proficiency, 
             Difference=fit.coef[, "Estimate"] - proficiency,
             Residual=(fit.coef[, "Estimate"] - proficiency)/fit.coef[, "Std. Error"])
hist(results[, "Residual"])
plot(results[, c("Actual", "Estimate")])

The histogram (lower left panel of the figure below) is of the differences between the estimated and actual proficiencies, expressed as multiples of the standard error of estimate. For a good procedure, these values will almost always lie between $-2$ and $2$ and be symmetrically distributed around $0$. With 1000 workers involved, though, we fully expect to see a few of these standardized differences to stretch out $3$ and even $4$ away from $0$. This is exactly the case here: the histogram is as pretty as one could hope for. (One might thing of course it's nice: these are simulated data, after all. But the symmetry confirms the weights are doing their job correctly. Using the wrong weights will tend to create an asymmetric histogram.)

The scatterplot (lower right panel of the figure) directly compares estimated proficiencies to actual ones. Of course this would not be available in reality, because we do not know the actual proficiencies: herein lies the power of the computer simulation. Observe:

  • If there had been no random variation in work (set cv=0 and rerun the code to see this), the scatterplot would be a perfect diagonal line. All estimates would be perfectly accurate. Thus, the scatter seen here reflects that variation.

  • Occasionally, an estimated value is pretty far away from the actual value. For instance, there is one point near (110, 160) where the estimated proficiency is about 50% greater than the actual proficiency. This is almost inevitable in any large batch of data. Bear this in mind if the estimates will be used on an individual basis, such as for evaluating workers. On the whole these estimates may be excellent, but to the extent the variation in work productivity is due to causes beyond any individual's control, then for a few of the workers the estimates will be erroneous: some too high, some too low. And there's no way to tell precisely who is affected.

Here are the four plots generated during this process.

Plots

Finally, note that this regression method is easily adapted to controlling for other variables that plausibly might be associated with group productivity. These could include the group size, the duration of each work effort, a time variable, a factor for the manager of each group, and so on. Just include them as additional variables in the regression.

$\endgroup$
  • $\begingroup$ Wow, thats a lot to take in. I guess I'm just not sure how I can tell who the hardest working workers are from these charts though. $\endgroup$ – Greg Guida Oct 5 '12 at 21:26
  • $\begingroup$ Are the charts on a per-worker basis? $\endgroup$ – Greg Guida Oct 5 '12 at 21:27
  • $\begingroup$ The lower right panel presents all 1,000 estimates. The highest one is around 200: it appears all the way to the right. The proficiency histogram and residual histogram also depict results for 1,000 workers. The upper right panel, a histogram of work, displays the total amounts of work for almost 5,000 jobs. $\endgroup$ – whuber Oct 5 '12 at 21:35
  • $\begingroup$ Ok, I get the meaning of each of the charts but I'm not sure how to use them to rank the workers. $\endgroup$ – Greg Guida Oct 5 '12 at 21:43
  • $\begingroup$ From top to bottom in the lower right panel. The code also creates a table of these results (called results): you can sort it by estimated value. You can export it to a spreadsheet, etc. $\endgroup$ – whuber Oct 5 '12 at 21:45
7
$\begingroup$

You'd want to set up your data something like this, with 1 indicating that the person was part of the team that did that row's work:

 work.done Alice Bob Carl Dave Eve Fred Greg Harry Isabel
 1.6631071     0   1    1    0   1    0    0     0      0
 0.7951651     1   1    0    0   0    0    0     1      0
 0.2650049     1   1    1    0   0    0    0     0      0
 1.2733771     0   0    0    0   1    0    0     1      1
 0.8086390     1   0    1    0   0    0    0     0      1
 1.7323428     1   0    0    0   0    0    1     0      1
 ...

Then, you can just do linear regression (assuming everything is additive, etc., as you mentioned in comments). In R, the command would be

lm(work.done ~ . + 0, data = my.data)

The "formula" work.done ~ . + 0 says, in English, that the amount of work done depends on all the other columns (that's the ".") and that groups with no workers would do no work (that's the "+ 0"). This will give you the approximate contribution from each worker to the average group output.

As discussed in the comments, if you have a pair of workers that are always together, the model won't be distinguish the contributions of the two workers from one another, and one of them will get an "NA".

$\endgroup$
  • $\begingroup$ Does it matter that there are 1000 workers? What does he mean by groups of 1-100? Even with the clarifications from the 2 edits I do not see where every data set identifies the individuals in the group? All I know is that each person works approximately the same amount each day. Since several of you think there is a solution possibly via regression explicitly what are the underlying assumptions and how is a person's work identifiable? Also I am puzzled about all the discussion of cooperative work. Nowhere is anything assumed other than working independently. $\endgroup$ – Michael R. Chernick Oct 5 '12 at 3:32
  • $\begingroup$ The only constraint I see is that each person somehow manages to do the same amount of work over a large number of trials? It seems like we are trying to translate that inot something sensible. But is it clear that this is what the OP intended?? $\endgroup$ – Michael R. Chernick Oct 5 '12 at 3:37
  • $\begingroup$ @MichaelChernick I'm not sure I understand your concern. Is it that the same individual could contribute different amounts on different trials, or is there more to it? $\endgroup$ – David J. Harris Oct 5 '12 at 16:41
  • $\begingroup$ @DavidHarris I think you have a nice solution if your assumptions are right. But I am worried about identifiability for each person with so many workers. The OP thinks the large sample size helps. But there needs to be structure like what you gave and some modeling assumptions. I just don't think he has specified everything we need to solve the problem. $\endgroup$ – Michael R. Chernick Oct 5 '12 at 16:49
  • $\begingroup$ @MichaelChernick I think that if we assume workers are independent, that a linear model is pretty safe, and the linearity also protects us from some problems we might run into. whuber is probably right about the weighting, which would help. Random effects for workers and groups could help keep the parameter estimates sane if there are identifiability problems. There are probably more improvements that could be made, but I still think this is on the right track, assuming the workers are essentially independent. $\endgroup$ – David J. Harris Oct 5 '12 at 18:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.