2
$\begingroup$

I’m looking for the best way to analyze a data set. The experiment was conducted at two different locations, X and Y. The experiment at location X was conducted in one year (2014), whereas the one conducted at location Y was repeated in the next year too (total two years – 2015 & 2016). There were a total of four treatments in this study A, B, C, D. Similar treatments (A, B, C) were compared at location Y. Location X did not have treatment A, but had an additional treatment D.
In summary, only treatments B, C were common at both locations. Similar treatments were compared at location Y, and treatment D was only tested at location X. What’s the best way to analyze this data?

  1. Should data be analyzed separately for each location? For location X, this is better because I can compare treatments B, C and D at location X without introducing any bias, as treatment D was not tested at location Y, so I will have the same number of data points to compare to compare treatments B, C and D. This is also better for location Y because similar treatments were compared and the experiment was repeated in the next year too. I will also be comparing similar number of data points for each treatment if data was analyzed separately at each location.

  2. Since treatments B & C were common at both locations, data should not be analyzed separately at both locations because I will get more data points to compare treatments B & C. The problem is that I will get uneven treatment number distribution in this case because treatment D was only tested at location X, so treatment D has only 23 data points, treatments B & C has 74 (23+25+26) data points, and treatment A has 51 (26 + 25) data points

I'm more inclined towards method 1 but looking for an expert opinion for confirmation. Thanks

enter image description here

Some details about my experiment. I put out plants in the field for a week to treat them with above four treatment, took them back to a controlled environment to count the number of infected leaves and DISCARDED them. I repeated the experiment with fresh plants in the next week. This is not a time series data. Treatments were applied for a week at both locations, so the duration of each treatment was the same. This is a positive count data. Plot size, treatment duration and sample material were identical. The only problem is uneven treatment distribution between 2014 & 2015-2016

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ What is the goal of your analysis? Other than the location, were the experiments otherwise identical (plot sizes, treatment durations, sample material etc)? $\endgroup$
    – George Savva
    Commented Jan 5, 2023 at 14:17
  • $\begingroup$ In what way might location affect the outcome data? $\endgroup$
    – George Savva
    Commented Jan 5, 2023 at 14:38

1 Answer 1

Reset to default
0
$\begingroup$

The different numbers of cases under different treatments isn't a problem on its own. Although there's a long historical preference in ANOVA for having equal numbers of cases in each treatment, a proper regression model will handle different numbers of cases.

The bigger problem is that you can't distinguish certain types of effects. For example, you can't distinguish differences between locations X and Y from differences between the calendar year 2014 versus years 2015-2016.

One way to deal with this is to model all of the data together and then do post-modeling comparisons of specific situations. That uses all of the data to build the model, but restricts subsequent analysis to comparisons that make sense given the limits of your experimental design.

Code each combination of location/treatment/year in a single 9-level categorical predictor: treatments B, C, D at X in 2014; treatments A, B, C at Y in 2015; treatments A, B, C at Y in 2016. You presumably will be using a Poisson or negative binomial model for the count data, and you should also be controlling for week within the year by including that in your model (for example, with a flexible regression spline). You might consider an interaction between your 9-level categorical predictor and your modeling of week within the year.

After you have fit the model, don't pay much attention to the individual coefficient values. Instead, use post-modeling tools like those provided by the R emmeans package to evaluate combinations of coefficients that provide useful information. For example, did the overall 2015 and 2016 estimates of treatment effects at location Y differ? What is the difference between treatments B and C, combining both locations? Between D and B/C at location X? Between A and B/C at location Y?

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for the detailed answer. I have marked the question as answered, but would appreciate if you could give me some feed back on what I have done. I have run the analyses for each location separately. I checked DHARMa package residuals for the model and there are no issue. I used Poisson family as the data was not over dispersed while controlling for year/week. Does this approach have any issues statistically? I am interested in comparing treatments only, not locations. $\endgroup$
    – Ahsk
    Commented Jan 9, 2023 at 14:07
  • $\begingroup$ @AhsaanKhan some might question use of year as a random effect when you only have (at most) 2 values for it at any one location. Might be better to use year as a fixed effect. The separate analysis by location might make your comparison of treatments B and C (done at both locations) a bit less powerful than in a combined analysis, but other comparisons should be pretty much the same as in the combined analysis as they are within single locations. So I don't see any major issues. Lack of over-dispersion and well-behaved residuals are good signs. $\endgroup$
    – EdM
    Commented Jan 9, 2023 at 14:27
  • $\begingroup$ My bad. Year should not have been included as a random effect. Thanks you very much. $\endgroup$
    – Ahsk
    Commented Jan 9, 2023 at 14:35
  • $\begingroup$ @AhsaanKhan it's not strictly required to treat predictors with small numbers of levels as fixed effects. That's why I said "some might question..." Sometimes the partial pooling provided by treating such predictors as random effects makes sense in multilevel models. See this page and its links for some discussion. The point is to find a way that describes your current data well and is likely to extend to other data samples. $\endgroup$
    – EdM
    Commented Jan 9, 2023 at 14:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.