This is a real experiment about to be performed

Batches of samples will be prepared. A Striker will be used to see if a reaction will occur. For example: 20 drops might be performed, and number of reactions out of 20 will be recorded. Several variables will be changed. Physics says the variables should affect reaction rates, hence why I'm including them in the model initially.

Will end up with data something like this:

  • 1 or 2 continuous variables for atmospheric data at time of strike
  • 2 or 3 categorical variables, probably binary, like which team prepared sample, etc
  • continuous variable for input reaction energy
  • and maybe a few other variables related to chemical properties, concentration etc

Thought #1

I can't decide what I want to use for the dependent variable. My initial thought was to compute the reaction rates for each set of trials. And use reaction rate as dependent variable.

The problem is that the number of trials per run might change. So some batches might not have the power to capture a good estimate.

Thought #2

My second thought was just to leave the drops unbatched. They are independent. Then I'd have a binary variable for reaction/noreaction for each trial.


For Thought #1 my approach would be to use either linear regression or logistic (leaning toward logistic due to large num of categorical and reaction rate likes a log transform)

For thought #2 I would use general linear models (Poisson process)

Does any of this sound like I'm on the right track? The experiment is actually defined (mostly) by an industry standard, so there is little I can do in terms of design modification.

I'm not a statistician, but I do have a good chunk of stat training under my belt, including graduate level theory classes in regression and computational stat.

A few other things of note: We don't care about forecasting, we simply want to look at how much change in the dependent variable can be accounted for by the predictors. And I use R, in case you're wondering.

I'm starting to wonder if I'm overthinking this whole thing and multi-level contingency table might suffice.


I've just been informed that based on time table and sample material available, the lab will not be able to perform a large amount of samples. I do not think it will be possible, for instance, to hold all but one IV static while the other is varied randomly. Because we will potentially have quite a few IVs. (But the number of IVs will still be small compared to number of trials) I'm not sure how to go about specifying a min sample. We don't know the probably of a react/noreact a priori. This is one reason for the testing.


1 Answer 1


For your Thought #1: Your dependent variable is obviously a binary variable reaction/noreaction of each trial as you mentioned or the pair of (number of reactions, number of trials) if you had batched trials.

For Thought #2: Let assume you don't have any limitation regarding to the number of experiments to run or the times you can use the lab.

To decide if to run experiments batched or unbatched: Batched runs would be a good option if you only had categorical variables in this research. So you could choose a combination of factor levels and do a batch experiment each time (i.e 20 run). But Here you have continuous variables as well, and this make your design a little more complicated.

Nevertheless if setting the variables' levels in your experimental environment is time consuming it is easier to set the explanatory variables in their levels (and choose random levels for continuous variables) and do a number of samples independently at a time (i.e 20 runs here - Batched drops as you say)

In any of the forms of running the experiment(batched or unbatched) there would be no problem with the analysis of data, I think. As you use R you can define a custom model and

For Batched trials > Use a binomial distribution for your dependent variable. in this way there would be no problem if you have different number of trials in each batch. you can use Maximum Likelihood estimation for estimating the effects and MLE's asymptotic distribution to test the hypothesis.

For UnBatched trials > Use a general linear model as you said(a logistic regression,probit regression, etc).

Finally consider that there would be some two-way interactions(or higher order of interactions) between the effect of your explanatory variables. If so, you may need a large number of runs to discover these effects. It's hard to say how many runs you will need for this, But if the first assumption is true, You can run your trials in a multi-phase approach. I mean you can start your experiment with a number of runs and do a preliminary analysis on your data. if you saw any sign on some potential significant interactions, then you can go back to the lab and continue the experiment with concentrating on the potential main effects and interactions (This will not recommend unless you can have the same environment with fix conditions in both phases of experiment).

  • $\begingroup$ would be a good option if you only had categorical variables in this research Well, here's the thing. The continuous variables will be pretty coarse. For example, we might have tons of numbers like 70, 71.2, 75.1, etc. But physics tells us that it might take +/- 100 before rate is affected by a measurable amount. So I was also kicking around the idea of binning the continuous variables to something like: "Below 100" "Above 100" etc. Again, according to our measuring ability, and physics references, this loss of 'resolution' shouldn't have an impact. $\endgroup$
    – Chad
    Mar 26, 2015 at 11:01
  • $\begingroup$ and do a number of samples independently at a time Each individual drop is independent. The experiment is completely reset between individual drops. So I'm correct in saying that the differing levels need not be batched at time of experiment, right? Post-hoc batching should be ok given the independence of trials? $\endgroup$
    – Chad
    Mar 26, 2015 at 11:03
  • $\begingroup$ would be some two-way interactions(or higher order of interactions After looking at prior data for this setup I can say there are DEFINITELY higher order interactions. I can see it in regression and with contingency table analysis. It's the kind of data where if you're a noob and just use stepwise to get the 'best' aic or r^2 you end up with a hilarious model a mile long full of permutations of interaction terms. (Don't worry, I know better than to do that) $\endgroup$
    – Chad
    Mar 26, 2015 at 11:04
  • $\begingroup$ @Chad yes, It's not needed to batch drops if they are completely independent. I'm not a physicist and don't have a clear imagination of what you are working on. But if setting the variables' levels and running the experiment is cheap and fast enough, there would be nothing to be worry about. just set the variables' levels randomly in each step and run an experiment. run the experiment as much as it's enough(you can have a good approximation of the number of runs from the prior data). $\endgroup$ Mar 26, 2015 at 12:14
  • $\begingroup$ You can use better model selection methods such as Lasso, Dantzig Selector, etc if you have a very complicated model consisting very high order of interactions. $\endgroup$ Mar 26, 2015 at 12:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.