Determining largest contributor in a group

Question

I don't know much about statistics so bear with me. Let's say I have a set of 1000 workers. I want to figure out who the hardest worker is but I can only measure the amount of work getting done in groups of 1-100 over an hours worth of work. Assuming each worker always does around the same amount of work, over a large number of trials and combinations can I rank my workers by who works hardest?

Note: this is just a metaphor, so don't worry about actually running the tests, just assume I already have a large set of data.

Edit: When I say "Assuming each worker always does around the same amount of work" I mean each individual does around the same amount of work on a day to day basis. So Joey will do around 100 units of work each day and Greg will do around 50. The problem is I can only observe the number of units of work done by the group.

More edits: In regards to the number of workers working at once and the frequency of them working. There could be any number of workers working at the same time. Some workers will probably end up working a lot more than others, that is to say, we can assume some workers will be working almost 90% of the time and others almost never.

I know that makes it difficult but I will have a very large dataset so hopefully that makes it a little bit easier.

For each hour we know which workers are working and how much work got done. From that information I want to find out who is doing the most work.

If the data were in JSON format it would look something like this:

[
  {
    "work_done": 12345,
    "Workers": [ "andy", "bob", "cameron", "david" ]
  },
  {
    "work_done": 432,
    "Workers": [ "steve", "joe", "andy"]
  },
  {
    "work_done": 59042,
    "Workers": [ "bob", "aaron", "michelle", "scott", "henry" ]
  },
  ...
]

Answer 1

David Harris has provided a great answer, but since the question continues to be edited, perhaps it would help to see the details of his solution. Highlights of the following analysis are:

• Weighted least squares is probably more appropriate than ordinary least squares.

• Because the estimates may reflect variation in productivity beyond any individual's control, be cautious about using them to evaluate individual workers.

---

To carry this out, let's create some realistic data using specified formulas so we can evaluate the accuracy of the solution. This is done with R:

set.seed(17)
n.names <- 1000
groupSize <- 3.5
n.cases <- 5 * n.names  # Should exceed n.names
cv <- 0.10              # Must be 0 or greater
groupSize <- 3.5        # Must be greater than 0
proficiency <- round(rgamma(n.names, 20, scale=5)); hist(proficiency)

In these initial steps, we:

• Set a seed for the random number generator so anybody can exactly reproduce the results.

• Specify how many workers there are with n.names.

• Stipulate the expected number of workers per group with groupSize.

• Specify how many cases (observations) are available with n.cases. (Later a few of these will be eliminated because they correspond, as it happens at random, to none of the workers in our synthetic workforce.)

• Arrange for the amounts of work to differ randomly from what would be predicted based on the sum of each group's work "proficiencies." The value of cv is a typical proportional variation; E.g., the $0.10$ given here corresponds to a typical 10% variation (which could range beyond 30% in a few cases).

• Create a workforce of people with varying work proficiencies. The parameters given here for computing proficiency create a range of over 4:1 between the best and worst workers (which in my experience may even be a bit narrow for technology and professional jobs, but perhaps is wide for routine manufacturing jobs).

With this synthetic workforce in hand, let's simulate their work. This amounts to creating a group of each workers (schedule) for each observation (eliminating any observations in which no workers at all were involved), summing the proficiencies of the workers in each group, and multiplying that sum by a random value (averaging exactly $1$) to reflect the variations that will inevitably occur. (If there were no variation at all, we would refer this question to the Mathematics site, where respondents could point out this problem is just a set of simultaneous linear equations which could be solved exactly for the proficiencies.)

schedule <- matrix(rbinom(n.cases * n.names, 1, groupSize/n.names), nrow=n.cases)
schedule <- schedule[apply(schedule, 1, sum) > 0, ]
work <- round(schedule %*% proficiency * exp(rnorm(dim(schedule)[1], -cv^2/2, cv)))
hist(work)

I have found it's convenient to put all the workgroup data into a single data frame for analysis but to keep the work values separate:

data <- data.frame(schedule)

This is where we would begin with real data: we would have the worker grouping encoded by data (or schedule) and the observed work outputs in the work array.

Unfortunately, if some workers are always paired, R's lm procedure simply fails with an error. We should check first for such pairings. One way is to find perfectly correlated workers in the schedule:

correlations <- cor(data)
outer(names(data), names(data), paste)[which(upper.tri(correlations) & 
                                             correlations >= 0.999999)]

The output will list pairs of always-paired workers: this can be used to combine these workers into groups, because at least we can estimate the productivity of each group, if not the individuals within it. We hope it just spits out character(0). Let's presume it does.

One subtle point, implicit in the foregoing explanation, is that the variation in the work performed is multiplicative, not additive. This is realistic: the variation in output of a large group of workers will, on an absolute scale, be greater than the variation in smaller groups. Accordingly, we will get better estimates by using weighted least squares rather than ordinary least squares. The best weights to use in this particular model are the reciprocals of the work amounts. (In the event some work amounts are zero, I fudge this by adding a small amount to avoid dividing by zero.)

fit <- lm(work ~ . + 0, data=data, weights=1/(max(work)/10^3+work))
fit.sum <- summary(fit)

This should take just one or two seconds.

Before going on we ought to perform some diagnostic tests of the fit. Although discussing those would take us too far afield here, one R command to produce useful diagnostics is

plot(fit)

(This will take a few seconds: it's a large dataset!)

Although these few lines of code do all the work, and spit out estimated proficiencies for each worker, we wouldn't want to scan through all 1000 lines of output--at least not right away. Let's use graphics to display the results.

fit.coef <- coef(fit.sum)
results <- cbind(fit.coef[, c("Estimate", "Std. Error")], 
             Actual=proficiency, 
             Difference=fit.coef[, "Estimate"] - proficiency,
             Residual=(fit.coef[, "Estimate"] - proficiency)/fit.coef[, "Std. Error"])
hist(results[, "Residual"])
plot(results[, c("Actual", "Estimate")])

The histogram (lower left panel of the figure below) is of the differences between the estimated and actual proficiencies, expressed as multiples of the standard error of estimate. For a good procedure, these values will almost always lie between $-2$ and $2$ and be symmetrically distributed around $0$. With 1000 workers involved, though, we fully expect to see a few of these standardized differences to stretch out $3$ and even $4$ away from $0$. This is exactly the case here: the histogram is as pretty as one could hope for. (One might thing of course it's nice: these are simulated data, after all. But the symmetry confirms the weights are doing their job correctly. Using the wrong weights will tend to create an asymmetric histogram.)

The scatterplot (lower right panel of the figure) directly compares estimated proficiencies to actual ones. Of course this would not be available in reality, because we do not know the actual proficiencies: herein lies the power of the computer simulation. Observe:

• If there had been no random variation in work (set cv=0 and rerun the code to see this), the scatterplot would be a perfect diagonal line. All estimates would be perfectly accurate. Thus, the scatter seen here reflects that variation.

• Occasionally, an estimated value is pretty far away from the actual value. For instance, there is one point near (110, 160) where the estimated proficiency is about 50% greater than the actual proficiency. This is almost inevitable in any large batch of data. Bear this in mind if the estimates will be used on an individual basis, such as for evaluating workers. On the whole these estimates may be excellent, but to the extent the variation in work productivity is due to causes beyond any individual's control, then for a few of the workers the estimates will be erroneous: some too high, some too low. And there's no way to tell precisely who is affected.

Here are the four plots generated during this process.

Finally, note that this regression method is easily adapted to controlling for other variables that plausibly might be associated with group productivity. These could include the group size, the duration of each work effort, a time variable, a factor for the manager of each group, and so on. Just include them as additional variables in the regression.

Answer 2

You'd want to set up your data something like this, with 1 indicating that the person was part of the team that did that row's work:

 work.done Alice Bob Carl Dave Eve Fred Greg Harry Isabel
 1.6631071     0   1    1    0   1    0    0     0      0
 0.7951651     1   1    0    0   0    0    0     1      0
 0.2650049     1   1    1    0   0    0    0     0      0
 1.2733771     0   0    0    0   1    0    0     1      1
 0.8086390     1   0    1    0   0    0    0     0      1
 1.7323428     1   0    0    0   0    0    1     0      1
 ...

Then, you can just do linear regression (assuming everything is additive, etc., as you mentioned in comments). In R, the command would be

lm(work.done ~ . + 0, data = my.data)

The "formula" work.done ~ . + 0 says, in English, that the amount of work done depends on all the other columns (that's the ".") and that groups with no workers would do no work (that's the "+ 0"). This will give you the approximate contribution from each worker to the average group output.

As discussed in the comments, if you have a pair of workers that are always together, the model won't be distinguish the contributions of the two workers from one another, and one of them will get an "NA".