4
$\begingroup$

I have data that were collected at a number of sites, and each site was located within one of three zones (Lake Ontario, Erie and the St. Lawrence), so I was hoping to do nested ANOVAs to compare between sites and zones. Unfortunately I have an unequal number of sites in each zone (3, 4 and 5 respectively) due to not enough data being collected at a couple sites. Also, my variances and sample sizes are also not equal between sites (sample size running from 3 to 43, I know, terrible!). The total number of observations for all sites was 182. Most sites had around 15 observations

My question is, is it possible for me to do nested analysis of some sort? I can't find much information on nested analysis with unequal variances.

I have tried transforming, the closest I can get to homoscedasticity is with $x_{new}=\frac{1}{(x+2)}$, and that gives me a $p \approx 0.02$

Improve this question
$\endgroup$
4
  • $\begingroup$ You can fit a mixed model with lme() in R or PROC MIXED in SAS. I do not understand your last sentence, what is X_new and x ? $\endgroup$
    – Stéphane Laurent
    Commented Apr 23, 2012 at 17:38
  • $\begingroup$ Sorry, I meant that I tried transforming my data to stabilize the variances, where X_new was the transformed data and x was the original data. I will try your suggestions. Do you know of anything similar in SPSS? I am trying to consistant in my program usage, but I will try the other two anyway. Thanks! $\endgroup$
    – Jess
    Commented Apr 23, 2012 at 17:49
  • 1
    $\begingroup$ As Stephane suggests, multilevel/mixed models are probably the way to go with this - no need to have equal sample sizes at each site nor equal numbers of sites in each zone. I listed a few resources in this old answer, of which I'd probably start with Gelman and Hill. There are also many other questions on this site about this kind of analysis - search for [multilevel-analysis], [mixed-models], and [repeated-measures]. $\endgroup$
    – Matt Parker
    Commented Apr 23, 2012 at 17:51
  • 1
    $\begingroup$ And here's an intro to mixed models in SPSS (PDF). $\endgroup$
    – Matt Parker
    Commented Apr 23, 2012 at 17:52

1 Answer 1

Reset to default
3
$\begingroup$

Don't do transformations for error control! Ditch the ANOVA and go with the linear model. Use robust standard errors to account for heteroscedasticity (unequal variances). Adjust for fixed effects for site by creating binary indicators for your three regions. How many observations per site and how many total sites? If the number of sites is relatively large relative to the total number of observations, trust robust standard errors to just give you a population averaged response.

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ Why not use transformations? It seems like a sensible approach to me. $\endgroup$
    – Michael Lew
    Commented Apr 23, 2012 at 23:45
  • $\begingroup$ Because transformations of data based on the error structure or, worse, based on the significance of your inference denigrate the credibility of your results. In scholarly journals, reviewers are quick to call out seemingly pointless transformations, e.g. those estimated by Box-Cox methods. It's a multiple testing issue. The problem is that the p-value no longer means what you think it means, it becomes very difficult to interpret since a repetition of the exact same experiment might lead you to a different transformation. $\endgroup$
    – AdamO
    Commented Apr 24, 2012 at 15:00
  • $\begingroup$ Thanks, I did feel a little uncertain about transforming since different transformations gave me different results and I wasn't sure how to interpret. I have between 3 and 43 observations per site, and 12 sites. The total number of observations for all sites was 182. Most sites had around 15 observations. $\endgroup$
    – Jess
    Commented Apr 24, 2012 at 17:29
  • 1
    $\begingroup$ @AdamO So the problem is with the possibility of trawling through a large list of possible transformations looking for 'significance' rather than with transformation of the data per se. In biomedical science there are many circumstances where transformation (mostly log transformation) makes sense because of the nature of the measurements. For example, the concentrations of biological molecules usually scale geometrically and so a log transform makes sense and controls heteroscedasticity. $\endgroup$
    – Michael Lew
    Commented Apr 26, 2012 at 21:24
  • 1
    $\begingroup$ I think I agree. A little bit of non-constant variance never hurt anyone, as Ray Carroll said. So fretting over weird funnel residual shapes seem worthless to me. Deciding a priori that a geometric mean difference is more interesting than an arithmetic mean difference is worth fretting over. However, deciding this based on the distribution of the data is a no-no. $\endgroup$
    – AdamO
    Commented Apr 26, 2012 at 21:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.