My experiment looks at the effect of 6 different treatments on various (usually 18) physical and chemical properties of the test material (silage), at several different time points. I do my statistical analysis using R. I had been using ANOVA (aov()) (with Tukey's test (HSD.test()) for post hoc analysis), but was often finding that my data was not normally distributed (using shapiro.test()) and/or had unequal variance (leveneTest() – car package). I therefore started testing all my data, regardless of normality, using Kruskal-Wallis test (kruskal() – from the agricolae package, which also gives a post hoc analysis…).

I usually present my data as mean values, with the standard deviation in parenthesis. I also include the result of the post hoc test as a letter to indicate statistically significant differences within a column.

Other researchers, such as Arriola et al 2011, present and analyse their data differently. They talk about using GLM in SAS (not R) as follows:

“The data were analyzed as a completely randomized design using the GLM procedure of SAS (v. 9.2 SAS Institute Inc., Cary, NC). The general model was Yij = μ + Ti + eij, where Yij = response variable, μ = overall mean, T = effect of treatment i, and eij = error term. The F-protected least significant difference test was used to compare least squares means and significance was declared at P < 0.05.”

I am unclear if the 'GLM' analysis they describe is a General Linear Model or a Generalized Linear Model (and also do not understand what the difference is). I also note that this description does not mention testing for normality of the data or homogeneity of variance. Is it the case that GLM, unlike ANOVA/Kruskal- Wallis, does not rely on assumptions regarding the normality and variance of the data? Also, they don’t present standard deviations, only reporting the mean (with letters to indicate significant differences between means), but give SEM (standard error of the mean?), and sometimes also P value.

So, my questions are as follows:

  1. Assuming I wish to replicate the analysis of Arriola et al above, ideally using R, which analysis (General Linear Model or Generalized Linear Model) should I be trying to do here?
  2. Does this GLM require normal data and homogeneity of variance? (I often find that my data are not normally distributed and can not easily be transformed to normality).
  3. Can I look for interactions (e.g. treatment vs time) using GLM, similar to that done using ANOVA?
  4. Does a GLM provide a post hoc type analysis to demonstrate which means are significantly different; something like the Tukey's HSD used with ANOVA? If not, can a post hoc analysis be applied to the result of the GLM?
  5. Can I do all of the above in R, allowing me to report mean values, along with SEM and a P value, as shown by Arriola et al?
  • 5
    $\begingroup$ Watch out: For different tribes, GLM means variously General Linear Models and Generalized Linear Models. They overlap but are not at all identical. So on #2 generalized linear models do not require either of those. In general, #1 is off-topic here and the second part of #2 is too open to answer (which researchers, which literature; it's always true that some statistical analyses are poorly explained and the main reasons why are mundane). As for #3, depends how much hock you drink, but, typo aside, this looks like an interesting answerable question if you expand it. $\endgroup$
    – Nick Cox
    Feb 22, 2018 at 16:45
  • $\begingroup$ Thanks for your comments @Nick. I have editied the question to try and clarify according to your suggestions. $\endgroup$
    – Mark
    Feb 28, 2018 at 13:09
  • $\begingroup$ Thanks for the attempt to clarify. Your new #5 is still off-topic. I am afraid that questions that expect people to study an external paper and digest a long several-part query don't often get much attention, so I bail out here myself, although even if you get partial replies that's better than none. $\endgroup$
    – Nick Cox
    Feb 28, 2018 at 13:32
  • $\begingroup$ I've never used SAS in my life, but a quick Google confirms "The GLM procedure uses the method of least squares to fit general linear models." $\endgroup$
    – Nick Cox
    Feb 28, 2018 at 13:54

2 Answers 2

  1. From the quote you present about the model used, what little I know about SAS, and a quick look at the paper, it doesn't seem that those authors used the SAS PROC GLM to do anything beyond what you would get from a standard analysis of variance (ANOVA). Although SAS PROC GLM can handle many types of experimental designs, even true multivariate analysis of multiple outcome variables as you have in silage analysis, the "general model" they display seems simply to be a set of standard ANOVAs comparing treatments one analyte at a time.

  2. As their "general model" seems to be a set of standard ANOVAs despite their use of PROC GLM, the standard requirements for statistical significance testing apply. You probably know that you don't need "normal data" to meet these requirements, just residuals close enough to normal/homoscedastic, but I point that out for others who might look at this page. Many of the technical analyses lead to results based on fractions/percentages (e.g. percentage of total weight that is dry matter, DM; percent of DM accounted for by different types of fiber, etc). Such data can pose difficulties in meeting the requirements with respect to residuals, as the values are restricted in range. As no values of this type seem to be exactly 0% or exactly 100%, working with a logit transform of the data in fractional form (that is, start with fractions in the range 0 to 1 rather than percentages, then take the logit) or beta regression might work better.

3-4. Based on the above, you don't need to go down the route toward SAS PROC GLM.

  1. The reported values for SEM and p seem to be based on the ANOVA within-cell mean square and the number of observations per cell. (It's not immediately clear how they dealt with their duplicate technical measurements on each of the 4 replicates of each treatment. In principle they could have had separate estimates for technical error and among-batch error, but that's not consistent with how they presented their "general model.")

Some quick responses. I think nothing I say contradicts what @EdM says.

1) GLM in SAS stands for "general linear model". The typical function for this in R is lm. For the example you present, this is not different than using aov. But lm is more flexible for some situations.

2) Yes, and you can look up the assumptions of general linear models. They include conditional normality and homoscedasticity. These are usually assessed by looking at the model residuals. There are other assumptions that you should be aware of.

3) Yes, a general linear model can include continuous IVs, nominal IVs, or interactions. The appropriate post-hoc analysis might depend upon what is included in the model. But, yes, they will be things like Tukey HSD.

4) You should be able to do all this in R. I'll defer to @EdM for the issue on the SEM. If you have good examples to follow, it is no more difficult to do this in R than conducting it in SAS. But you should understand that R is not written as a commercial software that spits out all the results you might want. Instead, you need to ask it for the results you want.

A few other notes:

a) It is fine to use Kruskal-Wallis with a Dunn test post-hoc for a design analogous to a one-way anova. I don't know what post-hoc the agricolae package uses, but I recommend using FSA::dunnTest for the post-hoc.

a2) But it will be worth your while to become familiar with conducting general linear models in R. You might look at: lm, car::Anova, emmeans.

b) I recommend against using tests to determine normality or homoscedasticity. You might see Normality tests don't do what you think they do.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.