What family of GLM when response is Bray-Curtis dissimilarity? Should I use "adonis" instead?

Question

I have some questions about testing for effects of different experimental levels on community similarity. I'll explain my planned experimental design before I ask the questions:

I have two methods for sampling species. I will use both of these methods on the same vegetation patch, at 3 points along a transect. The transect has a tree-line at one end, and I want to test whether the communities detected by the two methodologies become more different as one moves along the transect, away from the tree-line.

Further details of my experimental design: I will have 5 paired replicates (Method 1 and Method 2) at each distance from the tree-line. I will have 5 transects in a field. I will repeat this in five fields in different parts of the country (moving north to south).

I was thinking that as my response I would use Bray-Curtis dissimilarity of the communities detected by the two methods. My test would then be whether the dissimilarity changes with distance from the tree-line. I was thinking of coding the distances as "near", "middle", and "far", rather than as the numerical distance in cm.

I was thinking of using a generalized linear model in R to carry out my statistical test, and have a few questions:

I thought my model formula should be like this:

Bray-Curtis dissimilarities ~ Distance_from_tree-line + (1|transect) + (1|field)

All of the BC dissimilarities would be the response, and I would control for differences between transect and field. I'm quite new to R formulas, so wasn't sure if this was the correct terminology?

I will also be interested in looking for effects of field - will my formula above allow me to test for this effect, or should I change field to be an additional predictor?

Secondly, what type of GLM should I use if the response is a Bray-Curtis dissimilarity? They are somewhat similar to proportions, being bound between 0 and 1, so I thought I could use a binomial model. However, they are slightly different to normal proportions so I wasn't sure if this was appropriate. I've been looking this up, but haven't been able to find an answer.

Finally, I've seen that the adonis function in the vegan package uses model formulas, with a distance matrix as the response, so considered using this function to do my analysis. However, my response is a list of distances/dissimilarities - not a matrix. I'm not sure if I can implement this in adonis, does anyone have any thoughts?

Many thanks

Answer 1

Interesting experiment!

1) I do not think recoding distance from tree line to an ordinal variable is a good idea, you throw away valuable information and need to estimate an additional parameter, thereby reducing your power

2) Using the Bray-Curtis dissimilarity as outcome makes perfect sense to me

3) I would treat transect and field as normal variables:

Bray-Curtis dissimilarities ~ Distance_from_tree-line + transect + field

This way you test for significance of the transect and field variables. Always keep them in your model though, even when they are not significant because they are strata! Also testing for field*distance interactions may be interesting to see if the effect of the distance depends on the latitude:

Bray-Curtis dissimilarities ~ Distance_from_tree-line*field + transect

But this you need to decide for yourself based on "expert-knowledge"

4) A binomial response is not indicated here, that would only apply if you had two different outcomes, e.g. presence/absence data. It seems to me that normal linear regression will work (the lm() function in R), I assume the Bray-Curtis dissimilarity to be approximately normally distributed since it is a summary statistic of sorts