Help with analyzing/planning a physical experiment -- regression

Question

This is a real experiment about to be performed

Batches of samples will be prepared. A Striker will be used to see if a reaction will occur. For example: 20 drops might be performed, and number of reactions out of 20 will be recorded. Several variables will be changed. Physics says the variables should affect reaction rates, hence why I'm including them in the model initially.

Will end up with data something like this:

• 1 or 2 continuous variables for atmospheric data at time of strike
• 2 or 3 categorical variables, probably binary, like which team prepared sample, etc
• continuous variable for input reaction energy
• and maybe a few other variables related to chemical properties, concentration etc

Thought #1

I can't decide what I want to use for the dependent variable. My initial thought was to compute the reaction rates for each set of trials. And use reaction rate as dependent variable.

The problem is that the number of trials per run might change. So some batches might not have the power to capture a good estimate.

Thought #2

My second thought was just to leave the drops unbatched. They are independent. Then I'd have a binary variable for reaction/noreaction for each trial.

Regression

For Thought #1 my approach would be to use either linear regression or logistic (leaning toward logistic due to large num of categorical and reaction rate likes a log transform)

For thought #2 I would use general linear models (Poisson process)

Does any of this sound like I'm on the right track? The experiment is actually defined (mostly) by an industry standard, so there is little I can do in terms of design modification.

I'm not a statistician, but I do have a good chunk of stat training under my belt, including graduate level theory classes in regression and computational stat.

A few other things of note: We don't care about forecasting, we simply want to look at how much change in the dependent variable can be accounted for by the predictors. And I use R, in case you're wondering.

I'm starting to wonder if I'm overthinking this whole thing and multi-level contingency table might suffice.

---

Update

I've just been informed that based on time table and sample material available, the lab will not be able to perform a large amount of samples. I do not think it will be possible, for instance, to hold all but one IV static while the other is varied randomly. Because we will potentially have quite a few IVs. (But the number of IVs will still be small compared to number of trials) I'm not sure how to go about specifying a min sample. We don't know the probably of a react/noreact a priori. This is one reason for the testing.

Answer 1

For your Thought #1: Your dependent variable is obviously a binary variable reaction/noreaction of each trial as you mentioned or the pair of (number of reactions, number of trials) if you had batched trials.

For Thought #2: Let assume you don't have any limitation regarding to the number of experiments to run or the times you can use the lab.

To decide if to run experiments batched or unbatched: Batched runs would be a good option if you only had categorical variables in this research. So you could choose a combination of factor levels and do a batch experiment each time (i.e 20 run). But Here you have continuous variables as well, and this make your design a little more complicated.

Nevertheless if setting the variables' levels in your experimental environment is time consuming it is easier to set the explanatory variables in their levels (and choose random levels for continuous variables) and do a number of samples independently at a time (i.e 20 runs here - Batched drops as you say)

In any of the forms of running the experiment(batched or unbatched) there would be no problem with the analysis of data, I think. As you use R you can define a custom model and

For Batched trials > Use a binomial distribution for your dependent variable. in this way there would be no problem if you have different number of trials in each batch. you can use Maximum Likelihood estimation for estimating the effects and MLE's asymptotic distribution to test the hypothesis.

For UnBatched trials > Use a general linear model as you said(a logistic regression,probit regression, etc).

Finally consider that there would be some two-way interactions(or higher order of interactions) between the effect of your explanatory variables. If so, you may need a large number of runs to discover these effects. It's hard to say how many runs you will need for this, But if the first assumption is true, You can run your trials in a multi-phase approach. I mean you can start your experiment with a number of runs and do a preliminary analysis on your data. if you saw any sign on some potential significant interactions, then you can go back to the lab and continue the experiment with concentrating on the potential main effects and interactions (This will not recommend unless you can have the same environment with fix conditions in both phases of experiment).