# How should I design an experiment with non-binary factors?

We are preparing an experiment that will run during the coming winter, where we assess certain energy performance indicators on a set of single-family houses in Europe.

We are interested in finding out which factors have the greatest impact on these indicators. These factors will include the age of the building, its insulation level, its inertia, etc. I'm minded to treat these factors as, well, factors taking on a set of discrete values. For example, the age of the building will be coded as the decade in which it was built; the insulation level will be factored as poor, medium, good; etc.

I know more or less how to design an (fractional) factorial design when the factors can take on 2 values. But how should I design an experiment where, for the sake of the example, one factor can have 5 levels, another 3, and another 3?

Forcing continuous data into aggregate form loses you much statistical power, is all but certain to introduce bias (especially when nonlinear relationships exist), and induce logical fallacies if imputing biased relationships back to continuous phenomena.

So: why aggregate unnecessarily in the first place?

And: why not simply collect and analyze data on continuous phenomena as though they were continuous? For example, why not move to a regression context with continuous predictors?

Then, what you are aiming for is decent coverage of your measurements across the range of all your predictors of interest, particularly making sure you get enough observations in the extremes.

• Interesting point; however, how do I determine what a "decent coverage" means? Any pointers? – lindelof Sep 8 '14 at 15:23
• Relatively even spread of predictors, lack of gaps. You will also want to think very carefully about selection biases and/or about designed oversampling of that can be accounted for using, for example, weights. – Alexis Sep 8 '14 at 15:29
• @lindelof followup point: how do you determine "decent coverage" for discrete predictors? – Alexis Sep 10 '14 at 17:10

Assuming you are modeling a linear solution, the idea is to unfold each categorical variables (those with multiple levels) into a series of binary variables (aka dummy variables). For example, if you have a factor with three levels Low, Med, Hig, your linear regression problem will look like this:

$Y = \beta_0 + \beta_1Med + \beta_2Hig + e$

So Med will be set to 1 only when the observation level is Medium, else it will resolve to 0 (same for Hig). Note the absence of a Low level variable. That's the reference value for your categorical factor and its outcome will be implicit into the result of the equation (in this case the intercept will include the impact of Low, since it's the only other coefficient available).

• Interesting idea; and I suppose that the design should then disallow for "impossible" interactions such as Med = 1 and High = 1? Would you by any chance have any references? – lindelof Sep 8 '14 at 19:05
• Interactions make only sense across different variables. What is the interaction between two values of the same categorical variable? You can google categorical variables regression for more resources. I already included a link in my answer. – Robert Kubrick Sep 8 '14 at 19:51