testing significance in a specific setup

Question

Collective wisdom is desperately needed! I need to understand if some kind of significance testing is applicable here, and if that is the case - which test.

The data collection was devised as follows: There are 3 stations where people perform a number of station-specific manual tasks.

On each of three test days different people were assigned to stations (9 people in total). people were of different height, age and weight.

Each person (at each station on each day) was subjected to 5000 measurements of his kinetic activity, later classified as "good" or "bad". The percentages of "good" and "bad" for each person (and day and station) are thus known.

3 people from day one are treated as a "control group", or baseline.

I would like to know if a significance test is possible for percentages @ station[x] and day one (and baseline person working at that station on day one) and percentages @ same station for a different day/person.

stations    day1                     day2               day3
1           74.03%  (john)      74.83%  (jill)      75.44% (jorge)
2           45.83%  (baz)       80.24%  (basil)     76.45% (beata)
3           65.82%  (ted)       72.55%  (tessa)     72.73% (Tom)

percentages are proportions of "good" movements @ corresponding day and station. people working stations on day 1 weren't assisted by anything, people working stations on day 2 were given instructions, people working on day 3 were given visual aids.

Question: did the "ergonomic interference" influence proportion of "good" movements?

Null hypothesis is (I am guessing) "the difference can be attributed to natural variation in humans only" I want to test the significance of "had ergonomic instructions".

Can it be done with such a setup and which formula? Or is it a faulty experiment design and such testing is impossible?

Please help. I am no statistician and I am at my wits' end.

Answer 1

It sounds roughly like you are wondering if the proportion of measurements is predicted by training, after controlling for day, station, and individual (with interactions?). I'm unclear from your description how much replication you have for day, station, and individual; if your response is proportion of "good" motions, though, you only have 1 per person-station-day, even though you took 500 measurements. So you may be limited in what kind of model you can fit.

If my rough verbal description matches your ideas, I might consider logistic regression (a GLM or GLMM), and suggest Gelman and Hill 2007 Data Analysis Using Regression and Multilevel/Hierarchical Models as a resource.

Here's some ideas with some fake data that might help get you started.

#set up data frame 

people<-as.factor(c("a", "b", "c", "d", "e", "f", "g", "h", "i"))
stations<-as.factor(1:3)
days<-as.factor(1:5)
training<-as.factor(c("y", "n"))

#it is a problem with these kinds of numbers to try to estimate both the effect of people and of training, since they're kind of the same, but here's one way of reading your experimental design
dat<-expand.grid( stations, days,people)
names(dat)<-c( "stations", "days","people")
dat$training<-as.factor(c(rep("y",75), rep("n",60)))

#just making up # of good movements, and then changing the chance of success by constant multiplicative probability according to training
dat$success<-runif(length(dat$people), .3, 1)
dat[which(dat$training=="n"), "success"]<-dat[which(dat$training=="n"), "success"]*0.8
dat$success<-round(500*dat$success)
dat$failures<-500-dat$success

#one way to specify a glm when family= binomial is a matrix with success in one column, failures in another.
first.mod<-glm(cbind(dat$success, dat$failures)~stations+days+training+people, data=dat, family="binomial")

second.mod<-glm(cbind(dat$success, dat$failures)~stations+days+people, data=dat, family="binomial")

#these come out as exactly the same, but that's because of non-independence of people and training
summary(first.mod)
summary(second.mod)
anova(first.mod, second.mod)

#here's a  binomial regression model with no interactions, but a random effect of person
library(lme4)
third.mod<-glmer(cbind(dat$success, dat$failures)~stations+days+training+(1|people), data=dat, family="binomial")

#binned residual plots to check glm assumptions
library(arm)
binnedplot(predict(third.mod), resid(third.mod))

fourth.mod<-glmer(cbind(dat$success, dat$failures)~stations+days+(1|people), data=dat, family="binomial")
binnedplot(predict(fourth.mod), resid(fourth.mod))

#you might use a chi squared LR test to assess if training matters
anova(third.mod, fourth.mod)

#another link function might make more sense for you, like the poisson for count data, but again you're limited by data in my example where training and people are too collinear. Here people is a fixed effect again, but needn't be
fifth.mod<-glm(success~stations+days+training+people, data=dat, family="poisson")
sixth.mod<-glm(success~stations+days+people, data=dat, family="poisson")