I am a little bit confused about what an orthogonal design is and how it relates to the model matrix. It appears that there are many perspectives into the definition of orthogonal inside this post but none exactly helps me understand my problem. I am seeking an explanation with an experimental design context.

In Bailey 2008 pg 179, It introduces two factors G and F as orthogonal iff the subspaces $V_G \cap(V_F \cap V_G)^\bot$ and $V_F \cap(V_F \cap V_G)^\bot$ are both orthogonal to each other (or $V_G \cap V_{G \wedge F}^\bot$ and $V_F \cap V_{G \wedge F}^\bot$ is orthogonal)

A more intuitive theorem in the same book says (in terms of factors and levels) that

F and G on the same set are orthogonal to each other iff

  • every F-class meets every G-class
  • all these interesections have size proportional to the product of the sizes of the relevant F-class and G-class

However, the problem with these definitions is that it does help with 3 factors being orthogonal to each other or or when continuous covariates are included in your model. I was guessing that perhaps the model matrix when encoded can reveal something about orthogonality. However, my guess is that this is not true because It is obvious that when dummy coding the columns of the model matrix is not orthogonal to each other.

So my question is,

  1. What is a general definition of an orthogonal design? Is there a more general definition of orthogonal design? Including continuous covariates
  2. What are the advantages of orthogonal designs?
  3. Does the model matrix reveal anything about orthogonality?
  • 2
    $\begingroup$ I am not surprised that there is confusion about what is an Orthogonal Design. Orthogonal Designs were the subject of the 1979 book A.V.Geramita and Jennifer Seberry, Orthogonal Designs: Quadratic forms and Hadamard matrices, Marcel Dekker, New York - Basel, (1979), viii, 460 pages. This has now been republished as Jennifer Seberry, Orthogonal Designs, Springer Nature, 2017 $\endgroup$ Commented Jan 3, 2018 at 5:08
  • $\begingroup$ The orthogonality of two factors in a design is a concept subtler than people would think of. A mathematical rigorous and intuitive definition (which has similar flavor to the definition you listed in your question) of orthogonality of design is given in Section 2.3 of The Coordinate-Free Approach to Linear Models. Though it might be quite abstract at the first reading, this is the best exposition of orthogonal design I have ever encountered. If I got time, I would like to summarize the author's idea therein. $\endgroup$
    – Zhanxiong
    Commented Jan 3, 2018 at 5:25

1 Answer 1

  1. Perhaps you haven't fully grasped the definition yet. The requirements for orthogonal designs are that the blocking is orthogonal and the treatment is orthogonal. This simply means that crossproducts total to zero, whether the blocking or treatment is continuous, pseudo-continuous, polytomous, or binary. As @whuber correctly points out, statisticians often call dot products cross products, and furthermore often assume blocking and treatment factors have mean 0. So any blocking factor or treatment factor "crossed" with any other will come out to 0.

  2. Efficiency.

  3. Absolutely. We would expect that cross products between any two columns of the design matrix will total out to zero.

  • 4
    $\begingroup$ +1 In (2) it might perhaps be at least as important to list interpretability. $\endgroup$
    – whuber
    Commented Aug 8, 2016 at 15:28
  • $\begingroup$ @AdamO thanks for the reply. 1. I definitely haven't grasped the definition. Cross products of what exactly? 2. I am guessing that you mean dot product and not cross product (or do you really mean cross product). 3. Surely, one can make a design matrix (using dummy coding) such that dot product of two cols will not be zero. $\endgroup$ Commented Aug 8, 2016 at 16:33
  • 1
    $\begingroup$ @tintinthong I neglected to mention that one should consider all factors to be mean-centered. That is an (unfortunately) common omission. You're right about dot product. $\endgroup$
    – AdamO
    Commented Aug 8, 2016 at 16:53
  • 2
    $\begingroup$ tintin, in the regression context the term "cross product" often is used to refer to dot products: that is, a sum of squares or products. And yes, of course you can design a non-orthogonal experiment (almost surely a random design matrix will not be orthogonal): but that possibility seems to have no relevance to your questions. $\endgroup$
    – whuber
    Commented Aug 8, 2016 at 16:53
  • 2
    $\begingroup$ In addition to efficiency, if the design matrix is not orthogonal, there is no unique partition of the sums of squares. I think this is what @whuber means by interpretability: there would be different possible tests (w/ possibly different outcomes) & the data would not necessitate that 1 of them was the 'right' test. $\endgroup$ Commented Aug 8, 2016 at 16:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.